perf(llama-server): enable flash attention

Cuts prefill time with no accuracy cost, per markaicode.com's
llama.cpp timeout writeup — directly helps the prefill-vs-stream-idle
contention issue fixed in the prior two commits.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-05 11:32:17 +02:00
co-authored by Claude-Bot
parent 633292b291
commit 5767f548c3
+1
View File
@@ -20,6 +20,7 @@ services:
--n-gpu-layers ${LLAMA_GPU_LAYERS:-999}
--ctx-size ${LLAMA_CTX_SIZE:-131072}
--parallel ${LLAMA_PARALLEL:-2}
--flash-attn on
--jinja
# No published host port: llama-server is reached only via the omniroute
# gateway on the ai-stack docker network now — see issue #15. Its