feat(llm): switch to DFlash2 speculative decoding, drop qwen-classifier

Replace MTP with draft-dflash and a separate DFlash2 drafter GGUF.
Remove the qwen-classifier backend to free VRAM (GPU was at 100%).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
haylanandClaude Bot committed 2026-10-05 21:28:55 +02:00
1 parent 1e3ad169c0
commit 17747694bd
10 files changed
+30 -88

No files matched your search

+1 -1
View File
@@ -13,7 +13,7 @@ Browse to `http://<server>:<OMNIROUTE_DASHBOARD_PORT>` from the LAN. The dashboa
Each backend is registered once in the dashboard (Providers → add a **llama.cpp** provider) or via `POST /api/providers`. The base URL is `http://<service>:<container port>` on the stack network, using the service name and port from compose. None of this is checked into the repo. These docs assume the main model is registered as `qwen3.8-27b-local`.
- **`OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` / `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS`** have to be on, because backend URLs are private addresses. They relax OmniRoute's SSRF guard, which normally blocks registering a provider at a private or local address. Anyone with dashboard access can therefore point the gateway at any host on the stack network or the LAN.
- **Classifier connection:** turn OmniRoute's memory-tool injection off for it. Injected tools break the classifier's output ([`research/omniroute-memory-tool-injection-breaks-classifier.md`](research/omniroute-memory-tool-injection-breaks-classifier.md)). Also give it a generous per-connection timeout (see [Timeouts](#timeouts)).
- **Stale classifier connection:** the `qwen-classifier` backend no longer exists. Delete its provider connection in the dashboard, or repoint it at the main model. If repointed, turn OmniRoute's memory-tool injection off for it ([`research/omniroute-memory-tool-injection-breaks-classifier.md`](research/omniroute-memory-tool-injection-breaks-classifier.md)).
- Background on registering local backends: [`research/omniroute-qwen-websearch.md`](research/omniroute-qwen-websearch.md).
## API keys