Files

T

haylanandClaude-Bot 930e407053 docs(llm): document qwen-classifier reality, add --reasoning off safety net

- docker-compose.yml: add --reasoning off to qwen-classifier per
  fast-model-choice.md's own recommendation (ggml-org/llama.cpp#20809
  safety net) — missed in the original rollout, caught while writing
  this up.
- docs/coding-cli-setup/qwen-code.md: rewritten to match what's actually
  deployed (qwen-classifier, partial GPU offload, 65536 ctx, Q4_K_XL) —
  previously described an unimplemented llama-server-fast/8192-ctx plan.
  Documents the non-interactive MCP tool allow-list gap found live-testing.
- docs/coding-cli-setup/opencode.md: fix stale 65536 example that didn't
  match its own documented LLAMA_CTX_SIZE/LLAMA_PARALLEL formula (131072).
- docs/research/fast-model-choice.md: implementation note recording where
  the actual rollout diverged from this doc's original recommendations
  (service name, quant, context size, CPU-first-then-GPU path).
- docs/research/omniroute-account-semaphore-timeout.md: new — the
  hardcoded 30s per-connection semaphore timeout found during the
  pr-agent investigation, root-caused against OmniRoute's own source,
  and the maxConcurrent:null + providerSpecificData.timeoutMs fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-09 19:10:32 +02:00

2.1 KiB

Raw Permalink Blame History

OpenCode CLI

← back to overview

Confirmed project: anomalyco/opencode (renamed from sst/opencode — don't confuse with the unrelated opencode-ai/opencode Go TUI). Docs: https://opencode.ai/docs/

Install:

curl -fsSL https://opencode.ai/install | bash

Config (opencode.json, project root or ~/.config/opencode/opencode.json):

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "aiproxy": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "AI proxy (local)",
      "options": {
        "baseURL": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
        "apiKey": "<opencode-cli virtual key>"
      },
      "models": {
        "qwen3.8-27b-local": {
          "name": "Qwen3.8-27B",
          "limit": { "context": 131072, "output": 8192 }
        }
      }
    }
  }
}

Set limit.context to the per-slot context this stack actually serves — LLAMA_CTX_SIZE / LLAMA_PARALLEL from .env (262144 / 2 = 131072 by default), not raw LLAMA_CTX_SIZE and not a value assumed from the model card: llama.cpp divides --ctx-size across concurrent slots, so each request only gets one slot's share. OpenCode uses this for its own context-management bookkeeping, not the server.

Select the model with aiproxy/qwen3.8-27b-local.

OpenCode-specific risks (on top of the shared Qwen3.8-27B tool-calling risk — see overview):

Requires llama.cpp's --jinja flag (already set) — without it, OpenCode's unconditional tool-call scaffolding gets a 500.
anomalyco/opencode#20669 (closed as "not planned" — a live, unfixed risk): OpenCode's bash tool crashes if the model omits the optional description field on a tool call; some local backends return finish_reason: tool_calls with an empty array, which can hang the agent loop instead of stopping cleanly.
Thinking-mode handling (options.reasoningEffort) is undocumented for models that emit inline <think> tags rather than a native reasoning API field — expect no effect from that config on this model; untested.

Further reading: docs/research/opencode-cli-setup.md.

2.1 KiB Raw Permalink Blame History

OpenCode CLI

2.1 KiB

Raw Permalink Blame History