haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

Fix/omniroute pr agent timeout #54

Merged
haylan merged 2 commits from fix/omniroute-pr-agent-timeout into main 2026-09-09 14:29:13 +00:00
Conversation 0 Commits 2 Files Changed 5
+71 -69
haylan commented 2026-09-09 14:29:06 +00:00
Owner
Copy Link
Copy Source
No description provided.
haylan added 2 commits 2026-09-09 14:29:06 +00:00
feat(llm): dedicate a CPU-only backend for the qwen-code tool-call classifier 828bd4c046 
fastModel in ~/.qwen/settings.json (permissions.autoMode.classifier) was
aliased onto llama-server's own 27B connection, so every tool-call safety
check queued behind whatever heavy generation was already running on that
model's 2 GPU slots.

Add qwen-classifier: a separate llama.cpp instance, CPU-only, running
Qwen3-4B-Instruct-2507 (the smallest Qwen3 with native >=131072 context,
qwen-code's requirement, without lossy RoPE scaling). Structurally isolated
from llama-server's queue instead of sharing it. Sized for gameserver's
~17GiB free system RAM: q8_0/q8_0 KV at full 131072 ctx (~9.8GiB) + Q4_K_M-
class weights (~2.3GiB) fits comfortably, with better KV quality than the
q4_0 that would've been needed to fit this on the GPU's ~6GiB free VRAM.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
chroe(chore): added markdown for harnesses b3a64fe4b5
haylan merged commit c1e30ec9bb into main 2026-09-09 14:29:13 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
wayfinder:grilling
Wayfinder grilling ticket
 wayfinder:map
Wayfinder map issue
 wayfinder:prototype
Wayfinder prototype ticket
 wayfinder:research
Wayfinder research ticket
 wayfinder:task
Wayfinder task ticket
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
haylan (Arthur Erlich)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: haylan/LLM-Server#54