feat(llm): re-enable RAM prompt cache (8192 MiB) with --no-cache-idle-slots

--no-cache-idle-slots is the llama.cpp#27148 mitigation for --kv-unified,
which was the cause of misspelled output with --cache-ram on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
haylanandClaude Bot committed 2026-11-06 19:24:28 +01:00
1 parent 51684fb7a4
commit 2585f3eaab
3 files changed
+4 -3

No files matched your search

+2 -1
View File
@@ -27,7 +27,8 @@ services:
--flash-attn on
--cache-type-k ${LLAMA_KV_CACHE_TYPE:-q8_0}
--cache-type-v ${LLAMA_KV_CACHE_TYPE:-q8_0}
--cache-ram ${LLAMA_CACHE_RAM:-24576}
--cache-ram ${LLAMA_CACHE_RAM:-8192}
--no-cache-idle-slots
${LLAMA_SPEC_ARGS:-}
--jinja
# see docs/architecture.md#llama-server (flags), docs/network-access.md#backends (no host port)