docs(llm): document qwen-classifier reality, add --reasoning off safety net
- docker-compose.yml: add --reasoning off to qwen-classifier per fast-model-choice.md's own recommendation (ggml-org/llama.cpp#20809 safety net) — missed in the original rollout, caught while writing this up. - docs/coding-cli-setup/qwen-code.md: rewritten to match what's actually deployed (qwen-classifier, partial GPU offload, 65536 ctx, Q4_K_XL) — previously described an unimplemented llama-server-fast/8192-ctx plan. Documents the non-interactive MCP tool allow-list gap found live-testing. - docs/coding-cli-setup/opencode.md: fix stale 65536 example that didn't match its own documented LLAMA_CTX_SIZE/LLAMA_PARALLEL formula (131072). - docs/research/fast-model-choice.md: implementation note recording where the actual rollout diverged from this doc's original recommendations (service name, quant, context size, CPU-first-then-GPU path). - docs/research/omniroute-account-semaphore-timeout.md: new — the hardcoded 30s per-connection semaphore timeout found during the pr-agent investigation, root-caused against OmniRoute's own source, and the maxConcurrent:null + providerSpecificData.timeoutMs fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -79,6 +79,14 @@ services:
|
||||
# carried over from llama-server's comment above — same ROCm/ROCm#5706
|
||||
# clock-pinning bug applies now that two HIP contexts (this +
|
||||
# llama-server) share the card.
|
||||
#
|
||||
# --reasoning off is a no-cost safety net, not a confirmed-needed fix:
|
||||
# ggml-org/llama.cpp#20809 (closed) documents some server builds
|
||||
# misdetecting Qwen3-Instruct-2507 models as thinking models, routing
|
||||
# tool-call output into reasoning_content instead of tool_calls — exactly
|
||||
# the failure mode that ruled out the 27B model for this role in the
|
||||
# first place. Whether the current image build still has it was never
|
||||
# independently confirmed (see docs/research/fast-model-choice.md §4/§6).
|
||||
qwen-classifier:
|
||||
image: ghcr.io/ggml-org/llama.cpp:server-rocm
|
||||
container_name: qwen-classifier
|
||||
@@ -105,6 +113,7 @@ services:
|
||||
--batch-size 512
|
||||
--ubatch-size 128
|
||||
--flash-attn on
|
||||
--reasoning off
|
||||
--cache-type-k q4_0
|
||||
--cache-type-v q4_0
|
||||
--jinja
|
||||
|
||||
Reference in New Issue
Block a user