feat: remove llama-server-fast (Qwen3-4B classifier model) #51

2026-09-07T17:35:38Z

haylan commented

2026-09-07 17:35:38 +00:00

Removes the second always-resident llama.cpp instance (llama-server-fast, Qwen3-4B-Instruct-2507) and everything wired to it:

The service itself and its downloader-fast model-fetch job.
omniroute's depends_on: llama-server-fast.
LLAMA_FAST_* block in .env.example.
Stale VRAM-budget comment in scripts/switch-model.sh that assumed this service stayed resident during a swap.

⚠️ This was qwen-code's Auto Mode Stage 1 classifier (fastModel) — see docs/research/fast-model-choice.md and issue #44. Auto Mode loses that classifier until it's reconfigured to route elsewhere or falls back to prompt-only classification.

Not yet applied to the running R9700 stack — needs ./scripts/update.sh there after merge, per this repo's deploy convention.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DjhxWirQepKFEQj1huXNJR

Removes the second always-resident llama.cpp instance (`llama-server-fast`, Qwen3-4B-Instruct-2507) and everything wired to it: - The service itself and its `downloader-fast` model-fetch job. - `omniroute`'s `depends_on: llama-server-fast`. - `LLAMA_FAST_*` block in `.env.example`. - Stale VRAM-budget comment in `scripts/switch-model.sh` that assumed this service stayed resident during a swap. ⚠️ This was qwen-code's Auto Mode Stage 1 classifier (`fastModel`) — see `docs/research/fast-model-choice.md` and issue #44. Auto Mode loses that classifier until it's reconfigured to route elsewhere or falls back to prompt-only classification. Not yet applied to the running R9700 stack — needs `./scripts/update.sh` there after merge, per this repo's deploy convention. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DjhxWirQepKFEQj1huXNJR

haylan added 1 commit 2026-09-07 17:35:39 +00:00

feat: remove llama-server-fast (Qwen3-4B classifier model) 5d6a17fd9b

Drops the second always-resident llama.cpp instance, its downloader,
and the omniroute depends_on entry. Also strips the now-dead
LLAMA_FAST_* block from .env.example and the stale VRAM-budget comment
in scripts/switch-model.sh that assumed this service was always up.

Note: this was qwen-code's Auto Mode Stage 1 classifier (fastModel) —
see docs/research/fast-model-choice.md and issue #44. Auto Mode will
lose that classifier until/unless it's reconfigured to route
elsewhere or fall back to prompt-only classification.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DjhxWirQepKFEQj1huXNJR

haylan merged commit 9767261a96 into main

2026-09-07 17:56:27 +00:00

haylan deleted branch remove-fast-model

2026-09-07 17:56:27 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: haylan/LLM-Server#51