feat: remove llama-server-fast (Qwen3-4B classifier model)
Drops the second always-resident llama.cpp instance, its downloader, and the omniroute depends_on entry. Also strips the now-dead LLAMA_FAST_* block from .env.example and the stale VRAM-budget comment in scripts/switch-model.sh that assumed this service was always up. Note: this was qwen-code's Auto Mode Stage 1 classifier (fastModel) — see docs/research/fast-model-choice.md and issue #44. Auto Mode will lose that classifier until/unless it's reconfigured to route elsewhere or fall back to prompt-only classification. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DjhxWirQepKFEQj1huXNJR
This commit is contained in:
@@ -11,12 +11,6 @@
|
||||
# it never reliably sleeps a service on its own. A scripted swap always
|
||||
# knows which service should go up/down, so it doesn't need that heuristic.
|
||||
#
|
||||
# llama-server-fast (the small classifier model, issue #44) is NOT part of
|
||||
# this swap — it's meant to stay always-resident. Worst case with comfyui up
|
||||
# is comfyui (~25GB, Qwen-Image FP8) + llama-server-fast (~5GB) ≈ 30GB,
|
||||
# still under the 32GB card but tight — unverified on real hardware, check
|
||||
# `docker compose ps` / VRAM usage after the first real swap.
|
||||
#
|
||||
# OmniRoute may show the just-stopped provider as errored/offline in its
|
||||
# dashboard for up to CREDENTIAL_HEALTH_CHECK_INTERVAL (default 5 min) after
|
||||
# a swap — cosmetic, not a functional problem (see the research doc above).
|
||||
|
||||
Reference in New Issue
Block a user