feat(llm): re-enable RAM prompt cache (8192 MiB) with --no-cache-idle-slots
--no-cache-idle-slots is the llama.cpp#27148 mitigation for --kv-unified, which was the cause of misspelled output with --cache-ram on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
51684fb7a4
commit
2585f3eaab
3 files changed
+4
-3
No files matched your search
+1
-1
@@ -5,7 +5,7 @@ LLAMA_MODEL_FILE=Qwen3.8-27B-UD-Q4_K_XL.gguf
|
||||
LLAMA_GPU_LAYERS=999
|
||||
LLAMA_CTX_SIZE=262144
|
||||
LLAMA_PARALLEL=2
|
||||
LLAMA_CACHE_RAM=0
|
||||
LLAMA_CACHE_RAM=8192
|
||||
LLAMA_KV_CACHE_TYPE=q8_0
|
||||
|
||||
# Blank = speculative decoding off; see docs/architecture.md#llama-server
|
||||
|
||||
Reference in new issue
Block a user