feat(llm): make spec decoding and KV cache type .env toggles

Move all speculative-decoding flags into LLAMA_SPEC_ARGS (blank = off)
and the KV cache quant into LLAMA_KV_CACHE_TYPE, so token-corruption
A/B tests are a single .env edit instead of a compose edit.
Replaces LLAMA_DRAFT_MODEL_FILE and LLAMA_SPEC_N_MAX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
haylanandClaude Bot committed 2026-10-31 18:22:44 +01:00
1 parent 7d142ed59d
commit 8927e440aa
4 files changed
+10 -10

No files matched your search

+1 -1
View File
@@ -1,5 +1,5 @@
# Model files the downloader fetches into the models volume; see docs/operations.md#model-downloads
# `file` must match the LLAMA_*_FILE value in .env that loads it.
# `file` must match the name in .env that loads it (LLAMA_MODEL_FILE, or -md in LLAMA_SPEC_ARGS).
models:
- file: Qwen3.8-27B-UD-Q4_K_XL.gguf
url: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/Qwen3.8-27B-UD-Q4_K_XL.gguf