LLM-Server/docs/coding-cli-setup/qwen-code.md

Qwen Code CLI

← back to overview

Qwen Code speaks plain OpenAI Chat Completions, and — unlike the other CLIs — needs two models: the main chat model, and a fastModel for Auto Mode's action classifier (a separate, always-resident, always-fast instance so classification doesn't queue behind chat prefill; see docker-compose.yml's llama-server-fast service and docs/research/fast-model-choice.md). Both are registered as separate providers in OmniRoute but reachable through the same gateway URL. Config lives in ~/.qwen/settings.json:

{
  "modelProviders": {
    "openai": [
      {
        "id": "<main-model-provider-id-in-omniroute>",
        "name": "qwen3.8-27b-local",
        "envKey": "OMNIROUTE_API_KEY",
        "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
        "generationConfig": { "contextWindowSize": 131072 }
      },
      {
        "id": "<fast-model-provider-id-in-omniroute>",
        "name": "qwen3.8-27b-classifier",
        "envKey": "OMNIROUTE_API_KEY",
        "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
        "generationConfig": {
          "contextWindowSize": 8192,
          "extra_body": { "chat_template_kwargs": { "enable_thinking": false } }
        }
      }
    ]
  },
  "security": { "auth": { "selectedType": "openai" } },
  "model": {
    "name": "<main-model-provider-id-in-omniroute>",
    "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
  },
  "fastModel": "<fast-model-provider-id-in-omniroute>"
}

• envKey names the environment variable Qwen Code reads the virtual key from — set OMNIROUTE_API_KEY=<qwen-code-cli virtual key> before launching. Both providers can share one virtual key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls.
• contextWindowSize is per-slot, not LLAMA_CTX_SIZE itself — llama.cpp divides --ctx-size across LLAMA_PARALLEL concurrent slots, and each request only gets one slot's share (same correction applies to OpenCode's limit.context). Compute it per model from .env:
  • Main model: LLAMA_CTX_SIZE / LLAMA_PARALLEL = 262144 / 2 = 131072.
  • Fast model: LLAMA_FAST_CTX_SIZE / LLAMA_FAST_PARALLEL = 8192 / 1 = 8192. Undersizing this one specifically breaks Auto Mode ("Classifier stage 1 unavailable") once hints.allow/softDeny/hardDeny entries and recent-action history push a classifier call past it — see the LLAMA_FAST_CTX_SIZE comment in .env.example before raising it instead of LLAMA_FAST_PARALLEL.
• enable_thinking: false on the fast model matters: the fast model file (Qwen3-4B-Instruct-2507) is already non-thinking, but this also suppresses <think> output on any fast-model swap that isn't, keeping classifier responses parseable.
• Qwen Code also recognizes advisorModel, visionModel, compactionModel, imageModel for other model roles — none are wired up in this stack; only fastModel is required.

Web search via OmniRoute

Qwen Code's own built-in web search (tools.webSearch.enabled) has nothing to search with here — leave it false. Instead this stack's SearXNG-backed search (README §"Web search") is exposed through a thin stdio MCP wrapper around OmniRoute's /v1/search REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. ~/.qwen/mcp-servers/omniroute-search/index.mjs (needs @modelcontextprotocol/sdk and zod: npm init -y && npm i @modelcontextprotocol/sdk zod in that directory):

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const BASE_URL = process.env.OMNIROUTE_BASE_URL || "http://proxy-ai.home";
const API_KEY = process.env.OMNIROUTE_API_KEY;

if (!API_KEY) {
  console.error("OMNIROUTE_API_KEY is not set in the environment.");
  process.exit(1);
}

const server = new McpServer({ name: "omniroute-search", version: "1.0.0" });

server.registerTool(
  "search",
  {
    description: "Web/news search via OmniRoute's /v1/search endpoint.",
    inputSchema: { query: z.string().describe("Search query") },
  },
  async ({ query }) => {
    const res = await fetch(`${BASE_URL}/v1/search`, {
      method: "POST",
      headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` },
      body: JSON.stringify({ query }),
    });
    const text = await res.text();
    if (!res.ok) return { content: [{ type: "text", text: `HTTP ${res.status}: ${text}` }], isError: true };
    return { content: [{ type: "text", text }] };
  }
);

await server.connect(new StdioServerTransport());

Register it in ~/.qwen/settings.json:

{
  "mcpServers": {
    "omniroute-search": { "command": "node", "args": ["<path-to>/index.mjs"] }
  },
  "tools": { "webSearch": { "enabled": false } }
}

It reuses the same OMNIROUTE_API_KEY env var as the model providers above — the virtual key needs search permission in OmniRoute, not just chat-completions.

Auto Mode tuning

Auto Mode's action classifier calls the fast model above — its own request can queue behind other stack traffic before the fast llama-server instance is warm, so the default classifier timeout is worth raising. And since this stack is a single trusted local proxy, it's reasonable to pre-approve requests to it rather than confirm every call:

{
  "permissions": {
    "autoMode": {
      "classifier": { "timeouts": { "stage1Ms": 600000 } },
      "hints": { "allow": ["Requests to proxy-ai.home, my own local omniroute model proxy"] }
    }
  }
}

hints.allow entries are free-text descriptions the classifier matches against, not exact strings — capped at 150 entries/200 chars each (see the LLAMA_FAST_CTX_SIZE note above for why that ceiling matters).

Everything else in ~/.qwen/settings.json (hooks, security.auth's underlying tooling, editor prefs) is per-machine, not part of pointing at this stack — don't copy it wholesale between machines.