feat(llm): switch to DFlash2 speculative decoding, drop qwen-classifier
Replace MTP with draft-dflash and a separate DFlash2 drafter GGUF. Remove the qwen-classifier backend to free VRAM (GPU was at 100%). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
1e3ad169c0
commit
17747694bd
10 files changed
+30
-88
No files matched your search
+1
-1
@@ -175,7 +175,7 @@ docker compose build --pull
|
||||
|
||||
echo "==> ensuring models are downloaded (skips already-present files)"
|
||||
docker compose --profile tools run --rm downloader
|
||||
docker compose --profile tools run --rm downloader-classifier
|
||||
docker compose --profile tools run --rm downloader-draft
|
||||
|
||||
echo "==> bringing up omniroute"
|
||||
docker compose up -d --wait omniroute
|
||||
|
||||
Reference in new issue
Block a user