haylan/LLM-Server
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
150 Commits 1 Branch 0 Tags
68348aad726a5148f498cd81cdcdc552101c424f
Commit Graph
1 Commits
Author SHA1 Message Date
haylanandClaude Bot 408ed3e586 feat(llm): raise prompt cache RAM budget via --cache-ram 
Log shows 253 prompt-cache evictions and 146 full ~42k re-prefills at the default budget.
Add LLAMA_CACHE_RAM and the throughput-tuning research note.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
 2026-10-20 21:05:27 +02:00