# Qwen Code CLI [← back to overview](index.md) Qwen Code speaks plain **OpenAI Chat Completions**, and — unlike the other CLIs — needs *two* models: the main chat model, and a `fastModel` for Auto Mode's action classifier. Both are registered as separate providers in OmniRoute but reachable through the same gateway URL. ## Why a second model exists Auto Mode's action classifier (`permissions.autoMode`) is qwen-code's per-tool-call safety gate — it decides whether to auto-approve or block a shell command / tool call before it runs. It was originally aliased onto the main 27B model's own OmniRoute connection. That broke two ways in practice (see `docs/research/fast-model-choice.md` for the model research, and the issue-tracker history for the full incident): - **Queued behind heavy work.** Every classification call competed for the main model's 2 GPU slots with whatever real generation was already running, so a classifier check could sit blocked for minutes. - **CPU-only was tried first and was too slow.** Isolating the classifier onto its own CPU-only llama.cpp instance avoided the GPU queue entirely, but real classification calls (which can carry a non-trivial conversation transcript, not just the bare tool call) blew past OmniRoute's request timeout and retry-looped. The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.yml`) running a small model (`Qwen3-4B-Instruct-2507`) on its own **partial** GPU offload — enough layers on the R9700 to be fast, sized to leave real VRAM headroom next to the 27B model rather than trusting a naive weights+KV estimate (see that service's comment block in `docker-compose.yml` for the actual measured numbers and the two wrong turns — batch-size tuning, then flash-attn — before partial offload turned out to be the real lever). ## `~/.qwen/settings.json` ```json { "modelProviders": { "openai": [ { "id": "", "name": "qwen3.8-27b-local", "envKey": "OMNIROUTE_API_KEY", "baseUrl": "http://:${OMNIROUTE_PORT:-4000}/v1", "generationConfig": { "contextWindowSize": 131072 } }, { "id": "", "name": "qwen3-4b-classifier", "envKey": "OMNIROUTE_API_KEY", "baseUrl": "http://:${OMNIROUTE_PORT:-4000}/v1", "generationConfig": { "contextWindowSize": 65536, "extra_body": { "chat_template_kwargs": { "enable_thinking": false } } } } ] }, "security": { "auth": { "selectedType": "openai" } }, "model": { "name": "", "baseUrl": "http://:${OMNIROUTE_PORT:-4000}/v1" }, "fastModel": "" } ``` - `envKey` names the environment variable Qwen Code reads the virtual key from — set `OMNIROUTE_API_KEY=` before launching. Both providers can share one virtual key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls. - **`contextWindowSize` for the main model is per-slot, not `LLAMA_CTX_SIZE` itself** — llama.cpp divides `--ctx-size` across `LLAMA_PARALLEL` concurrent slots, and each request only gets one slot's share (same correction applies to OpenCode's `limit.context`). Compute it from `.env`: `LLAMA_CTX_SIZE / LLAMA_PARALLEL` = `262144 / 2` = **131072**. - **The classifier's `contextWindowSize` (65536) is not per-slot math** — `qwen-classifier` runs `--parallel 1`, so its whole `--ctx-size` belongs to the one slot. 65536 isn't a guess either: qwen-code's own source hard-caps the classifier transcript (`MAX_TRANSCRIPT_MESSAGES=40`, `MAX_HISTORICAL_ACTION_CHARS=4000`/message in `packages/core/src/permissions/classifier-transcript.ts`) — worst case is ~40-50K tokens, so 65536 gives real margin without wasting VRAM the way the original 131072 (copied from the main model's entry, not an actual qwen-code requirement) would have. - `enable_thinking: false` on the classifier matters for parseability, though `Qwen3-4B-Instruct-2507` is already architecturally non-thinking (see `fast-model-choice.md` §3) — this is belt-and-suspenders for any future fast-model swap that isn't. - Qwen Code also recognizes `advisorModel`, `visionModel`, `compactionModel`, `imageModel` for other model roles — none are wired up in this stack; only `fastModel` is required. ## Web search via OmniRoute Qwen Code's own built-in web search (`tools.webSearch.enabled`) has nothing to search with here — leave it `false`. Instead this stack's SearXNG-backed search (README §"Web search") is exposed through a thin stdio MCP wrapper around OmniRoute's `/v1/search` REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. `~/.qwen/mcp-servers/omniroute-search/index.mjs` (needs `@modelcontextprotocol/sdk` and `zod`: `npm init -y && npm i @modelcontextprotocol/sdk zod` in that directory): ```js import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; import { z } from "zod"; const BASE_URL = process.env.OMNIROUTE_BASE_URL || "http://proxy-ai.home"; const API_KEY = process.env.OMNIROUTE_API_KEY; if (!API_KEY) { console.error("OMNIROUTE_API_KEY is not set in the environment."); process.exit(1); } const server = new McpServer({ name: "omniroute-search", version: "1.0.0" }); server.registerTool( "search", { description: "Web/news search via OmniRoute's /v1/search endpoint.", inputSchema: { query: z.string().describe("Search query") }, }, async ({ query }) => { const res = await fetch(`${BASE_URL}/v1/search`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` }, body: JSON.stringify({ query }), }); const text = await res.text(); if (!res.ok) return { content: [{ type: "text", text: `HTTP ${res.status}: ${text}` }], isError: true }; return { content: [{ type: "text", text }] }; } ); await server.connect(new StdioServerTransport()); ``` Register it in `~/.qwen/settings.json`: ```json { "mcpServers": { "omniroute-search": { "command": "node", "args": ["/index.mjs"] } }, "tools": { "webSearch": { "enabled": false } } } ``` It reuses the same `OMNIROUTE_API_KEY` env var as the model providers above — the virtual key needs search permission in OmniRoute, not just chat-completions. **Non-interactive mode (`qwen -p ...`) needs this tool explicitly allow-listed.** MCP tools require interactive confirmation by default; `--approval-mode auto` alone doesn't bypass that for a non-interactive run — pass `--allowed-tools mcp__omniroute-search__search` (or `-y` for full YOLO) alongside `-p`, or the search call never reaches the classifier at all and silently no-ops. Confirmed live: without the allow-list, only the tool calls the CLI's non-interactive gate lets through end up as classifier requests. ## Auto Mode tuning Auto Mode's action classifier calls the fast model above. Even on the dedicated GPU-resident instance, give it real timeout headroom rather than trusting OmniRoute's default — and since this stack is a single trusted local proxy, it's reasonable to pre-approve requests to it rather than confirm every call: ```json { "permissions": { "autoMode": { "classifier": { "timeouts": { "stage1Ms": 600000 } }, "hints": { "allow": ["Requests to proxy-ai.home, my own local omniroute model proxy"] } } } } ``` `hints.allow` entries are free-text descriptions the classifier matches against, not exact strings — capped at 150 entries/200 chars each. Also set a generous per-connection timeout on the classifier's own OmniRoute provider connection (`providerSpecificData.timeoutMs`, dashboard or `PATCH /api/providers/{id}` — not a `.env` value, see `docs/network-access.md` for reaching the dashboard API). 120000ms is comfortable for the current GPU-resident setup (real measured latency: well under a second for a short check, low seconds for the largest realistic transcript) — this isn't the 20-minute figure the main 27B connection needs, since the classifier isn't competing for a contended GPU slot the way the main model can. Everything else in `~/.qwen/settings.json` (`hooks`, `security.auth`'s underlying tooling, editor prefs) is per-machine, not part of pointing at this stack — don't copy it wholesale between machines.