feat(llm): switch to DFlash2 speculative decoding, drop qwen-classifier
Replace MTP with draft-dflash and a separate DFlash2 drafter GGUF. Remove the qwen-classifier backend to free VRAM (GPU was at 100%). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
1e3ad169c0
commit
17747694bd
10 files changed
+30
-88
No files matched your search
@@ -22,7 +22,7 @@ Terms as defined in [`CONTEXT.md`](../CONTEXT.md). The stack doesn't run its own
|
||||
|
||||
## Backends
|
||||
|
||||
`llama-server` and `qwen-classifier` publish no host port (`expose` only). They have no authentication, so they're reachable only on the stack's Docker network, through the gateway. Making them directly reachable would put an unauthenticated inference endpoint on the network. That's a deliberate security change, not a config tweak.
|
||||
`llama-server` publishes no host port (`expose` only). It has no authentication, so it's reachable only on the stack's Docker network, through the gateway. Making it directly reachable would put an unauthenticated inference endpoint on the network. That's a deliberate security change, not a config tweak.
|
||||
|
||||
## qdrant
|
||||
|
||||
|
||||
Reference in new issue
Block a user