haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(llm): point qwen-classifier at the Q8 GGUF already on disk

Browse Source 
The Q4_K_M-class file this originally specced didn't exist yet on
gameserver (classifier crash-looped: "No such file or directory").
A Q8_K_XL GGUF for the same model was already sitting in the models
volume from something earlier — point at that instead of downloading a
new file, and drop the KV cache quant to q4_0/q4_0 to keep total RAM
comfortable now that the weights are the larger Q8 variant.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
haylan
2026-09-09 16:37:54 +02:00
co-authored by Claude-Bot
parent b3a64fe4b5
commit 20cc0bcc70
2 changed files with 14 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+2 -4
View File
@@ -45,10 +45,8 @@ LLAMA_PARALLEL=2
# Dedicated CPU-only backend for qwen-code's tool-call harmfulness classifier
# (fastModel in ~/.qwen/settings.json) — see docker-compose.yml's
# qwen-classifier service comment for the why. 131072 ctx / q8_0 KV / Q4_K_M-
# class weights ≈ 12GiB, fits gameserver's ~17GiB free system RAM with room
# to spare.
LLAMA_CLASSIFIER_MODEL_FILE=Qwen3-4B-Instruct-2507-UD-Q4_K_XL.gguf
# qwen-classifier service comment for the why and the RAM math.
LLAMA_CLASSIFIER_MODEL_FILE=Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf
# --- Lazytainer ---
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.