LLM-Server/docs/research/omniroute-qwen-websearch.md

OmniRoute + Qwen Code CLI web search — setup research

Investigates how to (a) confirm/complete OmniRoute's routing to this stack's local Qwen model, and (b) enable Qwen Code CLI's web-search tool, for a user running qwen from WSL against this repo's docker-compose stack.

What's already configured (verified live in WSL, 2026-09-05)

Checked via wsl.exe -- bash -lc '...' against ~/.qwen/:

• qwen-code CLI is installed: which qwen → /home/haylan/.local/bin/qwen, qwen --version → 0.23.0.
• ~/.qwen/settings.json already points at this stack's OmniRoute gateway, in the exact shape OmniRoute's own setup-qwen command produces (see below):

  "modelProviders": {
    "openai": [
      {
        "id": "qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf",
        "name": "qwen3.8-27b-local",
        "envKey": "OMNIROUTE_API_KEY",
        "baseUrl": "http://proxy-ai.home/v1",
        "generationConfig": { "contextWindowSize": 131072 }
      }
    ]
  },
  "security": { "auth": { "selectedType": "openai" } },
  "model": {
    "name": "qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf",
    "baseUrl": "http://proxy-ai.home/v1"
  }
  

  This targets http://proxy-ai.home/v1 (this repo's OmniRoute gateway hostname per docs/network-access.md), reads the API key from the OMNIROUTE_API_KEY env var, and matches docs/coding-cli-setup.md's convention of naming the registered provider qwen3.8-27b-local. Two backup files (settings.json.bak-cbm-*, settings.json.save) show earlier iterations of the same config — this was set up deliberately, not a stray default.
  • Not a gap — verified correct: contextWindowSize: 131072 matches LLAMA_CTX_SIZE / LLAMA_PARALLEL (262144 / 2), not half of it. docker-compose.yml (lines 21–22) runs llama-server with --ctx-size ${LLAMA_CTX_SIZE:-262144} --parallel ${LLAMA_PARALLEL:-2}, and .env.example (line 39) spells out that each of the two concurrent slots gets LLAMA_CTX_SIZE / LLAMA_PARALLEL tokens — i.e. 131072 per slot, matching commit 23e90fe ("cap concurrent slots at 2"). So ~/.qwen/settings.json's value is correctly sized to what one slot actually offers; no fix needed here.
  • Unverified: whether OMNIROUTE_API_KEY is actually set in the WSL environment or in a ~/.qwen/.env file — env | grep -i qwen in the same session showed no OMNIROUTE_API_KEY in the current shell (only PATH entries matched qwen), and ~/.qwen/.env wasn't checked (missed in the executed probe — see Open questions). If it's unset, qwen calls will fail auth against OmniRoute regardless of the web_search setup below.
• No web-search config exists yet: env | grep -i tavily and env | grep -i search both returned nothing; settings.json has no tools.webSearch key and no mcpServers entry for Tavily/Bailian/GLM search or for OmniRoute's own MCP server (it does have an unrelated mcpServers.codebase-memory-mcp stdio entry for this repo's own codebase-memory tool). grep -ril "tavily\|websearch\|web_search\|web-search" ~/.qwen matched only unrelated project chat-log files (from an unrelated shopware-420-seeds project), not any config.
• Conclusion: model routing (a) is already done. Web search (b) is not configured at all — no API key, no MCP server, no built-in-tool setting.

(a) OmniRoute → local Qwen model routing

Sources: this repo's docker-compose.yml (lines 63–133) and .env.example (lines 54–90); README.md §"AI gateway (OmniRoute)"; docs/coding-cli-setup.md; OmniRoute's own docs at github.com/mckazzy/OmniRoute-run-qwen, ref release/v3.8.50.

Current repo state: docker-compose.yml's omniroute service comment (lines 63–67) states routing is registered "once through the dashboard or POST /api/providers after first boot, not checked into this repo." OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true and OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS=true are already set (lines 84–88) so the dashboard/API will accept llama-server's container-internal address instead of rejecting it as a private URL.

Confirmed against OmniRoute's own reference docs (docs/reference/ENVIRONMENT.md at the pinned ref):

• OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS — default false; the doc says it is "REQUIRED for self-hosted providers" (it names LM Studio, Ollama, vLLM, Llamafile, Triton, SearXNG). Confirms the repo's own comment is correct and necessary.
• OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS — default true ("local-first"); false would block localhost/LAN/private ranges outright (cloud-metadata IPs stay blocked either way).
• OMNIROUTE_WS_BRIDGE_SECRET — "REQUIRED in production — when unset, all WS bridge requests are rejected," generated via openssl rand -base64 32 — matches this repo's comment (lines 89–93) and scripts/update.sh autofill.

docs/reference/PROVIDER_REFERENCE.md (same ref) lists llama-cpp as a built-in "Local, self-hosted" provider ID:

"Configure the OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1)"

This is a good match for this stack's llama-server container, which exposes port 8080 only on the internal ai-stack Docker network (docker-compose.yml lines 30–33, "No published host port"). Inside that network the service is reachable by its Compose service name, so the base URL to register should be http://llama-server:8080/v1, not 127.0.0.1 (127.0.0.1 inside the OmniRoute container would mean OmniRoute itself, not llama-server — they're different containers on the same bridge network).

Concrete steps (dashboard, matching docs/proxy-key-onboarding.md's existing pattern for reaching the dashboard):

1. Reach the dashboard: from the R9700 box, docker inspect -f '{{.NetworkSettings.Networks.ai_stack.IPAddress}}' omniroute, then browse http://<that-ip>:20128; from elsewhere, SSH-tunnel ssh -L 20128:<container-ip>:20128 <host> then browse localhost:20128.
2. Providers → Add provider → llama.cpp (llama-cpp provider ID per PROVIDER_REFERENCE.md).
3. Set base URL to http://llama-server:8080/v1 (the Compose service name — both containers share the ai-stack network per docker-compose.yml's networks: [ai-stack] on both services). No API key needed (llama-server's endpoint is unauthenticated internally, per docs/network-access.md).
4. Register the model under that provider using the naming this repo already assumes downstream (qwen3.8-27b-local, per docs/coding-cli-setup.md line 8) — pick a model ID/name here and keep it consistent everywhere a CLI config references it (~/.qwen/settings.json's existing entry already assumes this name).
5. Mint or reuse a virtual API key for the qwen-code-cli workload per docs/proxy-key-onboarding.md (label qwen-code-cli), and confirm it's the value behind OMNIROUTE_API_KEY in the WSL environment (or ~/.qwen/.env — see Open questions) that ~/.qwen/settings.json's envKey references.

OmniRoute's own automation for this exact CLI — docs/guides/CLI-INTEGRATIONS.md at the pinned ref documents a dedicated omniroute setup-qwen command:

omniroute setup-qwen --model qwen/qwen3.8-max-preview — writes ~/.qwen/settings.json (V4 modelProviders.openai array) and stores OMNIROUTE_API_KEY in ~/.qwen/.env; supports --yes (non-interactive), --config-path / --env-path (custom locations), and works in local or remote mode.

The ~/.qwen/settings.json found on this machine has exactly the V4 modelProviders.openai shape this command produces, and the two .bak/.save files back that up — this was very likely already run once, pointed at whichever model ID was registered in the dashboard at the time (the id field embeds the GGUF filename, qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf, matching .env.example's LLAMA_MODEL_FILE). Re-running it after registering/renaming the provider in step 2–4 above is the fastest way to refresh this file if the registered model ID ever changes (contextWindowSize: 131072 itself is already correct — see note above on --parallel).

(b) Qwen Code CLI web search

Sources: qwenlm.github.io/qwen-code-docs/en/developers/tools/web-search/, .../en/developers/tools/mcp-server/, .../en/users/configuration/settings/; OmniRoute's docs/frameworks/MCP-SERVER.md and docs/reference/PROVIDER_REFERENCE.md at release/v3.8.50.

Qwen Code's web-search docs page states plainly: the original built-in web_search tool ("Tavily/Google/GLM/DashScope multi-provider") "and its configuration were removed." Current options, per that same page:

1. New built-in web_search tool — DashScope-only now, not multi-provider. Needs tools.webSearch.enabled: true and tools.webSearch.model (e.g. "qwen3.6-plus") in settings.json, or equivalent env vars if settings.json can't be edited; requires a DASHSCOPE_API_KEY (Alibaba Cloud). It "issues a self-contained search request to a small auxiliary model with DashScope's server-side web_search (and web_extractor) tools, and returns the narrated findings plus source URLs" — i.e. it calls out to Alibaba's cloud, not this stack's local model or SearXNG.
   • Caveat: users/configuration/settings/ (the canonical settings-schema page) does not list tools.webSearch anywhere among its documented tools.* keys — only tools.sandbox, tools.shell, tools.core, tools.exclude, tools.disabled. This key may be genuinely undocumented there, or newer than that page's last update. Treat tools.webSearch as unconfirmed against the settings schema itself — verify with qwen --help or by testing once a DASHSCOPE_API_KEY is available (see Open questions).
2. MCP-based search — three named services: Alibaba Cloud Bailian WebSearch, Tavily WebSearch, GLM WebSearch Prime — each added as an mcpServers entry in settings.json. Confirmed schema from developers/tools/mcp-server/: HTTP/SSE servers use httpUrl (or url for SSE) plus an optional headers object, e.g.:

   { "mcpServers": { "tavily": {
       "httpUrl": "https://mcp.tavily.com/mcp/?tavilyApiKey=${TAVILY_API_KEY}"
   } } }
   

   (stdio servers instead use command/args/env/cwd, as the existing codebase-memory-mcp entry in this machine's ~/.qwen/settings.json does.)

Neither of Qwen Code's own two paths uses this stack's existing SearXNG integration. But OmniRoute — already in front of this stack's model — has its own MCP server with a built-in multi-provider web-search tool, and this repo already wires SearXNG through OmniRoute (README.md §"Web search": "The gateway also fronts SearXNG-backed web search"; .env.example's SEARXNG_LAN_IP / search.home extra_hosts entry in docker-compose.yml lines 106–109). OmniRoute's docs/frameworks/MCP-SERVER.md (pinned ref):

"Web search through OmniRoute search gateway (Serper/Brave/Perplexity/Exa/Tavily/Google PSE/Linkup/SearchAPI/SearXNG) with failover" — exposed as an omniroute_web_search tool requiring the execute:search scope.

And docs/reference/PROVIDER_REFERENCE.md lists searxng-search as one of its 12 built-in search-provider IDs: "API key is optional. Set your SearXNG base URL. Some instances may require a bearer token for access." — meaning SearXNG can be registered as a search provider in the OmniRoute dashboard the same way llama-cpp is registered as a model provider, no separate API key needed for a self-hosted SearXNG instance.

This means the path that reuses what's already deployed in this stack (SearXNG, already reachable from OmniRoute via search.home) is: connect qwen-code to OmniRoute's MCP server, not to Tavily/DashScope/GLM directly. Concrete steps:

1. In the OmniRoute dashboard, register SearXNG as a search provider (searxng-search), pointing at http://search.home (already resolvable inside the OmniRoute container via the extra_hosts entry in docker-compose.yml). This may already be done — README.md implies the gateway already fronts SearXNG-backed search, but confirm live in the dashboard since, per the same docker-compose.yml comment (lines 63–67), provider registration isn't checked into this repo.
2. Mint an API key scoped for MCP search use — OmniRoute's MCP-SERVER.md names execute:search (to actually call the search tool) and mcp:connect (narrow, MCP-connect-only) as the relevant scopes; manage/admin also work but are broader than needed.
3. Add an mcpServers entry to ~/.qwen/settings.json pointing at OmniRoute's MCP endpoint, using the same httpUrl/headers shape Qwen Code already supports for Tavily:

   {
     "mcpServers": {
       "omniroute-search": {
         "httpUrl": "http://proxy-ai.home/api/mcp/stream",
         "headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
       }
     }
   }
   

   (proxy-ai.home matches the hostname the model-provider entry already uses in this same file; swap in whatever host:port actually fronts OmniRoute's API port from WSL — docs/network-access.md says proxy-ai.home points at ${OMNIROUTE_PORT:-4000}, the API port, and docker-compose.yml/.env.example separately track OMNIROUTE_API_PORT (default 20129) as the container-internal port — confirm which one NPM actually proxies to before trusting the /api/mcp/stream path resolves through proxy-ai.home unchanged; this wasn't independently verified against a live instance, see Open questions.)
4. Set OMNIROUTE_SEARCH_KEY in the WSL shell profile (or in ~/.qwen/.env, consistent with how setup-qwen already stores OMNIROUTE_API_KEY there).
5. Restart qwen; the model should now see an MCP tool for web search backed by this stack's own SearXNG, routed and rate-limited the same way its LLM calls already are.

If instead the goal is simply "get any web search working fastest, reuse nothing," the plain Tavily-MCP or DashScope built-in-tool paths above are simpler (one API key, no dashboard provider registration) — but they bypass this stack's OmniRoute/SearXNG setup entirely and send queries to an external paid API instead.

Follow-up verification (2026-09-05, live checks)

• OMNIROUTE_API_KEY — confirmed set and working. It's exported from ~/.bashrc (line 133), not ~/.qwen/.env — invisible to a non-interactive bash -lc probe because .bashrc's standard top-of-file guard (case $- in *i*) ;; *) return;; esac) skips the rest of the file for non-interactive shells; a real interactive shell (bash -ic, or wsl + qwen as actually run) sources it fine. Verified: curl -H "Authorization: Bearer $OMNIROUTE_API_KEY" http://proxy-ai.home/v1/models → 200. Routing (a) is confirmed end-to-end, no action needed.
• MCP endpoint located from primary source — fetched OmniRoute's docs/frameworks/MCP-SERVER.md at release/v3.8.50 directly. Resolves the port ambiguity above: the MCP server runs on port 20128 (dashboard/API port), paths /api/mcp/stream (streamable HTTP), /api/mcp/sse, and /api/mcp/status. It states: /api/mcp/* is in OmniRoute's LOCAL_ONLY authz tier (src/server/authz/routeGuard.ts) — loopback-only by default; a non-loopback client needs a key carrying the manage scope or the narrower mcp:connect scope (added v3.8.0), and the server's Settings must have mcpEnabled on with the matching mcpTransport selected. omniroute_web_search itself additionally needs execute:search. No separate "MCP key type" exists — same provider keys, different scopes.
• Live probe result: curl http://proxy-ai.home:20128/api/mcp/status returns {"error":{"code":"AUTH_001","message":"Authentication required"}} identically with or without the Authorization: Bearer $OMNIROUTE_API_KEY header — the existing model-routing key isn't recognized on this route at all, consistent with it lacking mcp:connect/manage/execute:search scope and/or mcpEnabled not yet being turned on in the dashboard. This is dashboard-side state (not in git, no session credentials available from this environment) — genuinely needs a human with dashboard access, not another probe.
• Config prepared to unblock as soon as that's done: added an omniroute-search entry to ~/.qwen/settings.json's mcpServers (backed up the prior file first as settings.json.bak-wayfinder-<timestamp>):

  "omniroute-search": {
    "httpUrl": "http://proxy-ai.home:20128/api/mcp/stream",
    "headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
  }
  

  Deliberately a separate env var (OMNIROUTE_SEARCH_KEY), not reusing OMNIROUTE_API_KEY, so the search-scoped key stays distinct from the model-routing key — matches docs/proxy-key-onboarding.md's per-workload key pattern.

Resolution (2026-09-05, completed)

The dashboard steps above turned out to need a different diagnosis than originally guessed — walked through live with a oma_live_... management token and a rotating set of sk-... provider keys the user supplied:

• /api/providers (management API) showed zero search providers at all — not a misconfigured searxng-search entry, it simply didn't exist as a connection anymore (9 connections total, all LLM/chat providers). The CHANGELOG at the pinned ref was checked and shows /v1/search under active feature development (a feat(search) entry adding Firecrawl support), so this wasn't an OmniRoute-side removal of the search system — the row was just gone from this instance's own database (reason unconfirmed: update migration vs. prior manual removal).
• Created it via the API, not the dashboard UI — POST /api/providers turned out to accept the same generic connection schema used for LLM providers: {"provider":"searxng-search","name":"searxng"} (Zod-validated; an empty-body POST surfaced the required fields). Then PATCH /api/providers/<id> with {"providerSpecificData":{"baseUrl":"http://search.home/search"}} set the real URL, replacing the catalog default.
• Verified end-to-end: POST /v1/search with provider: "searxng-search" returned real results (5 hits, search_cost_usd: 0, cached: false, response_time_ms: 4495) — confirms search.home's extra_hosts mapping in docker-compose.yml resolves correctly from inside the OmniRoute container and the whole chain (OmniRoute → SearXNG → results) works.
• /api/mcp/status confirmed scopesEnforced: false on this instance — the mcp:connect/execute:search scope requirement documented upstream isn't actually being enforced here, so any valid provider key connects. mcpEnabled: true already, transport streamable-http.
• Key rotation caveat hit live: the first sk-... key the user shared went from working to a flat 401 on every route (including /v1/models) partway through testing — consistent with it having been revoked/rotated server-side. A second key worked immediately. If this setup stops working later, check for exactly this before re-diagnosing the whole chain.
• Final ~/.qwen/settings.json mcpServers entry (confirmed connected via qwen mcp list → ✓ omniroute-search: ... - Connected):

  "omniroute-search": {
    "httpUrl": "http://proxy-ai.home/api/mcp/stream",
    "headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
  }
  

  OMNIROUTE_SEARCH_KEY is exported in ~/.bashrc alongside the existing OMNIROUTE_API_KEY, holding the second (working) sk-... key.

Status: done. qwen in WSL has a connected omniroute-search MCP server backed by this stack's own SearXNG instance — no external search API, no Alibaba DashScope dependency. Not yet exercised: an actual qwen chat turn that triggers the omniroute_web_search tool call (only the MCP handshake and the raw /v1/search call were verified directly).

Open questions / unverified

• tools.webSearch.* settings.json schema — described on Qwen Code's web-search doc page but absent from the canonical settings-schema page; not independently confirmed (e.g. via qwen --help or source) — moot for this setup since the MCP path (above) is what's being wired in, not the DashScope-only built-in tool.
• DashScope vs SearXNG data-residency/cost tradeoffs — out of scope here, but worth noting the built-in web_search tool sends queries to Alibaba Cloud regardless of this stack being otherwise fully self-hosted.
• OmniRoute's own docs (already flagged in this repo's README.md) describe stealth/anti-detection and TLS-interception features elsewhere in its repo; none of that is exercised by anything in this note, but it's the same caveat README.md already carries forward from issue #31.