LLM-Server/docs/research/omniroute-qwen-websearch.md

# OmniRoute + Qwen Code CLI web search — setup research

Investigates how to (a) confirm/complete OmniRoute's routing to this stack's local
Qwen model, and (b) enable Qwen Code CLI's web-search tool, for a user running
`qwen` from WSL against this repo's docker-compose stack.

## What's already configured (verified live in WSL, 2026-09-05)

Checked via `wsl.exe -- bash -lc '...'` against `~/.qwen/`:

- **qwen-code CLI is installed**: `which qwen` → `/home/haylan/.local/bin/qwen`, `qwen --version` → `0.23.0`.
- **`~/.qwen/settings.json` already points at this stack's OmniRoute gateway**, in the exact shape OmniRoute's own `setup-qwen` command produces (see below):
  ```json
  "modelProviders": {
    "openai": [
      {
        "id": "qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf",
        "name": "qwen3.8-27b-local",
        "envKey": "OMNIROUTE_API_KEY",
        "baseUrl": "http://proxy-ai.home/v1",
        "generationConfig": { "contextWindowSize": 131072 }
      }
    ]
  },
  "security": { "auth": { "selectedType": "openai" } },
  "model": {
    "name": "qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf",
    "baseUrl": "http://proxy-ai.home/v1"
  }
  ```
  This targets `http://proxy-ai.home/v1` (this repo's OmniRoute gateway hostname per `docs/network-access.md`), reads the API key from the `OMNIROUTE_API_KEY` env var, and matches `docs/coding-cli-setup.md`'s convention of naming the registered provider `qwen3.8-27b-local`. Two backup files (`settings.json.bak-cbm-*`, `settings.json.save`) show earlier iterations of the same config — this was set up deliberately, not a stray default.
  - **Not a gap — verified correct**: `contextWindowSize: 131072` matches `LLAMA_CTX_SIZE / LLAMA_PARALLEL` (`262144 / 2`), not half of it. `docker-compose.yml` (lines 21–22) runs llama-server with `--ctx-size ${LLAMA_CTX_SIZE:-262144} --parallel ${LLAMA_PARALLEL:-2}`, and `.env.example` (line 39) spells out that each of the two concurrent slots gets `LLAMA_CTX_SIZE / LLAMA_PARALLEL` tokens — i.e. 131072 per slot, matching commit `23e90fe` ("cap concurrent slots at 2"). So `~/.qwen/settings.json`'s value is correctly sized to what one slot actually offers; no fix needed here.
  - **Unverified**: whether `OMNIROUTE_API_KEY` is actually set in the WSL environment or in a `~/.qwen/.env` file — `env | grep -i qwen` in the same session showed no `OMNIROUTE_API_KEY` in the *current* shell (only `PATH` entries matched `qwen`), and `~/.qwen/.env` wasn't checked (missed in the executed probe — see Open questions). If it's unset, `qwen` calls will fail auth against OmniRoute regardless of the `web_search` setup below.
- **No web-search config exists yet**: `env | grep -i tavily` and `env | grep -i search` both returned nothing; `settings.json` has no `tools.webSearch` key and no `mcpServers` entry for Tavily/Bailian/GLM search or for OmniRoute's own MCP server (it does have an unrelated `mcpServers.codebase-memory-mcp` stdio entry for this repo's own codebase-memory tool). `grep -ril "tavily\|websearch\|web_search\|web-search" ~/.qwen` matched only unrelated project chat-log files (from an unrelated `shopware-420-seeds` project), not any config.
- **Conclusion**: model routing (a) is already done. Web search (b) is not configured at all — no API key, no MCP server, no built-in-tool setting.

## (a) OmniRoute → local Qwen model routing

Sources: this repo's `docker-compose.yml` (lines 63–133) and `.env.example`
(lines 54–90); `README.md` §"AI gateway (OmniRoute)"; `docs/coding-cli-setup.md`;
OmniRoute's own docs at `github.com/mckazzy/OmniRoute-run-qwen`, ref
`release/v3.8.50`.

**Current repo state**: `docker-compose.yml`'s `omniroute` service comment (lines 63–67)
states routing is registered "once through the dashboard or `POST /api/providers`
after first boot, not checked into this repo." `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true`
and `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS=true` are already set (lines 84–88) so the
dashboard/API will accept `llama-server`'s container-internal address instead of
rejecting it as a private URL.

**Confirmed against OmniRoute's own reference docs**
(`docs/reference/ENVIRONMENT.md` at the pinned ref):
- `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` — default `false`; the doc says it is
  **"REQUIRED for self-hosted providers"** (it names LM Studio, Ollama, vLLM,
  Llamafile, Triton, SearXNG). Confirms the repo's own comment is correct and necessary.
- `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS` — default `true` ("local-first"); `false`
  would block localhost/LAN/private ranges outright (cloud-metadata IPs stay
  blocked either way).
- `OMNIROUTE_WS_BRIDGE_SECRET` — "REQUIRED in production — when unset, all WS
  bridge requests are rejected," generated via `openssl rand -base64 32` — matches
  this repo's comment (lines 89–93) and `scripts/update.sh` autofill.

`docs/reference/PROVIDER_REFERENCE.md` (same ref) lists **`llama-cpp`** as a
built-in "Local, self-hosted" provider ID:

> "Configure the OpenAI-compatible base URL (default: `http://127.0.0.1:8080/v1`)"

This is a good match for this stack's `llama-server` container, which exposes
port 8080 only on the internal `ai-stack` Docker network (`docker-compose.yml`
lines 30–33, "No published host port"). Inside that network the service is
reachable by its Compose service name, so the base URL to register should be
`http://llama-server:8080/v1`, not `127.0.0.1` (127.0.0.1 inside the OmniRoute
container would mean OmniRoute itself, not llama-server — they're different
containers on the same bridge network).

**Concrete steps** (dashboard, matching `docs/proxy-key-onboarding.md`'s
existing pattern for reaching the dashboard):

1. Reach the dashboard: from the R9700 box, `docker inspect -f
   '{{.NetworkSettings.Networks.ai_stack.IPAddress}}' omniroute`, then browse
   `http://<that-ip>:20128`; from elsewhere, SSH-tunnel
   `ssh -L 20128:<container-ip>:20128 <host>` then browse `localhost:20128`.
2. Providers → Add provider → **llama.cpp** (`llama-cpp` provider ID per
   `PROVIDER_REFERENCE.md`).
3. Set base URL to `http://llama-server:8080/v1` (the Compose service name — both
   containers share the `ai-stack` network per `docker-compose.yml`'s `networks:
   [ai-stack]` on both services). No API key needed (llama-server's endpoint is
   unauthenticated internally, per `docs/network-access.md`).
4. Register the model under that provider using the naming this repo already
   assumes downstream (`qwen3.8-27b-local`, per `docs/coding-cli-setup.md` line 8)
   — pick a model ID/name here and keep it consistent everywhere a CLI config
   references it (`~/.qwen/settings.json`'s existing entry already assumes this name).
5. Mint or reuse a virtual API key for the `qwen-code-cli` workload per
   `docs/proxy-key-onboarding.md` (label `qwen-code-cli`), and confirm it's the
   value behind `OMNIROUTE_API_KEY` in the WSL environment (or `~/.qwen/.env` —
   see Open questions) that `~/.qwen/settings.json`'s `envKey` references.

**OmniRoute's own automation for this exact CLI** — `docs/guides/CLI-INTEGRATIONS.md`
at the pinned ref documents a dedicated `omniroute setup-qwen` command:

> `omniroute setup-qwen --model qwen/qwen3.8-max-preview` — writes
> `~/.qwen/settings.json` (V4 `modelProviders.openai` array) and stores
> `OMNIROUTE_API_KEY` in `~/.qwen/.env`; supports `--yes` (non-interactive),
> `--config-path` / `--env-path` (custom locations), and works in local or remote mode.

The `~/.qwen/settings.json` found on this machine has exactly the V4
`modelProviders.openai` shape this command produces, and the two `.bak`/`.save`
files back that up — this was very likely already run once, pointed at whichever
model ID was registered in the dashboard at the time (the `id` field embeds the
GGUF filename, `qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf`, matching
`.env.example`'s `LLAMA_MODEL_FILE`). Re-running it after registering/renaming
the provider in step 2–4 above is the fastest way to refresh this file if the
registered model ID ever changes (`contextWindowSize: 131072` itself is already
correct — see note above on `--parallel`).

## (b) Qwen Code CLI web search

Sources: `qwenlm.github.io/qwen-code-docs/en/developers/tools/web-search/`,
`.../en/developers/tools/mcp-server/`, `.../en/users/configuration/settings/`;
OmniRoute's `docs/frameworks/MCP-SERVER.md` and `docs/reference/PROVIDER_REFERENCE.md`
at `release/v3.8.50`.

**Qwen Code's web-search docs page states plainly**: the *original* built-in
`web_search` tool ("Tavily/Google/GLM/DashScope multi-provider") **"and its
configuration were removed."** Current options, per that same page:

1. **New built-in `web_search` tool** — DashScope-only now, not multi-provider.
   Needs `tools.webSearch.enabled: true` and `tools.webSearch.model` (e.g.
   `"qwen3.6-plus"`) in `settings.json`, or equivalent env vars if `settings.json`
   can't be edited; requires a `DASHSCOPE_API_KEY` (Alibaba Cloud). It "issues a
   self-contained search request to a small auxiliary model with DashScope's
   server-side `web_search` (and `web_extractor`) tools, and returns the
   narrated findings plus source URLs" — i.e. it calls out to Alibaba's cloud,
   not this stack's local model or SearXNG.
   - **Caveat**: `users/configuration/settings/` (the canonical settings-schema
     page) does **not** list `tools.webSearch` anywhere among its documented
     `tools.*` keys — only `tools.sandbox`, `tools.shell`, `tools.core`,
     `tools.exclude`, `tools.disabled`. This key may be genuinely undocumented
     there, or newer than that page's last update. Treat `tools.webSearch` as
     unconfirmed against the settings schema itself — verify with `qwen --help`
     or by testing once a `DASHSCOPE_API_KEY` is available (see Open questions).
2. **MCP-based search** — three named services: Alibaba Cloud Bailian WebSearch,
   Tavily WebSearch, GLM WebSearch Prime — each added as an `mcpServers` entry
   in `settings.json`. Confirmed schema from `developers/tools/mcp-server/`:
   HTTP/SSE servers use `httpUrl` (or `url` for SSE) plus an optional `headers`
   object, e.g.:
   ```json
   { "mcpServers": { "tavily": {
       "httpUrl": "https://mcp.tavily.com/mcp/?tavilyApiKey=${TAVILY_API_KEY}"
   } } }
   ```
   (stdio servers instead use `command`/`args`/`env`/`cwd`, as the existing
   `codebase-memory-mcp` entry in this machine's `~/.qwen/settings.json` does.)

**Neither of Qwen Code's own two paths uses this stack's existing SearXNG
integration.** But OmniRoute — already in front of this stack's model — has its
own MCP server with a **built-in multi-provider web-search tool**, and this
repo already wires SearXNG through OmniRoute (`README.md` §"Web search":
"The gateway also fronts SearXNG-backed web search"; `.env.example`'s
`SEARXNG_LAN_IP` / `search.home` extra_hosts entry in `docker-compose.yml`
lines 106–109). OmniRoute's `docs/frameworks/MCP-SERVER.md` (pinned ref):

> "Web search through OmniRoute search gateway
> (Serper/Brave/Perplexity/Exa/Tavily/Google PSE/Linkup/SearchAPI/SearXNG) with
> failover" — exposed as an `omniroute_web_search` tool requiring the
> `execute:search` scope.

And `docs/reference/PROVIDER_REFERENCE.md` lists `searxng-search` as one of its
12 built-in search-provider IDs: **"API key is optional. Set your SearXNG base
URL. Some instances may require a bearer token for access."** — meaning
SearXNG can be registered as a search provider in the OmniRoute dashboard the
same way `llama-cpp` is registered as a model provider, no separate API key
needed for a self-hosted SearXNG instance.

**This means the path that reuses what's already deployed in this stack (SearXNG,
already reachable from OmniRoute via `search.home`) is: connect qwen-code to
OmniRoute's MCP server, not to Tavily/DashScope/GLM directly.** Concrete steps:

1. In the OmniRoute dashboard, register SearXNG as a search provider
   (`searxng-search`), pointing at `http://search.home` (already resolvable
   inside the OmniRoute container via the `extra_hosts` entry in
   `docker-compose.yml`). This may already be done — `README.md` implies the
   gateway already fronts SearXNG-backed search, but confirm live in the
   dashboard since, per the same `docker-compose.yml` comment (lines 63–67),
   provider registration isn't checked into this repo.
2. Mint an API key scoped for MCP search use — OmniRoute's `MCP-SERVER.md`
   names `execute:search` (to actually call the search tool) and `mcp:connect`
   (narrow, MCP-connect-only) as the relevant scopes; `manage`/`admin` also work
   but are broader than needed.
3. Add an `mcpServers` entry to `~/.qwen/settings.json` pointing at OmniRoute's
   MCP endpoint, using the same `httpUrl`/`headers` shape Qwen Code already
   supports for Tavily:
   ```json
   {
     "mcpServers": {
       "omniroute-search": {
         "httpUrl": "http://proxy-ai.home/api/mcp/stream",
         "headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
       }
     }
   }
   ```
   (`proxy-ai.home` matches the hostname the model-provider entry already uses
   in this same file; swap in whatever host:port actually fronts OmniRoute's API
   port from WSL — `docs/network-access.md` says `proxy-ai.home` points at
   `${OMNIROUTE_PORT:-4000}`, the *API* port, and `docker-compose.yml`/`.env.example`
   separately track `OMNIROUTE_API_PORT` (default `20129`) as the
   container-internal port — confirm which one NPM actually proxies to before
   trusting the `/api/mcp/stream` path resolves through `proxy-ai.home` unchanged;
   this wasn't independently verified against a live instance, see Open questions.)
4. Set `OMNIROUTE_SEARCH_KEY` in the WSL shell profile (or in `~/.qwen/.env`,
   consistent with how `setup-qwen` already stores `OMNIROUTE_API_KEY` there).
5. Restart `qwen`; the model should now see an MCP tool for web search backed by
   this stack's own SearXNG, routed and rate-limited the same way its LLM calls
   already are.

If instead the goal is simply "get *any* web search working fastest, reuse
nothing," the plain Tavily-MCP or DashScope built-in-tool paths above are
simpler (one API key, no dashboard provider registration) — but they bypass
this stack's OmniRoute/SearXNG setup entirely and send queries to an external
paid API instead.

## Follow-up verification (2026-09-05, live checks)

- **`OMNIROUTE_API_KEY` — confirmed set and working.** It's exported from
  `~/.bashrc` (line 133), *not* `~/.qwen/.env` — invisible to a non-interactive
  `bash -lc` probe because `.bashrc`'s standard top-of-file guard
  (`case $- in *i*) ;; *) return;; esac`) skips the rest of the file for
  non-interactive shells; a real interactive shell (`bash -ic`, or `wsl` +
  `qwen` as actually run) sources it fine. Verified: `curl -H "Authorization:
  Bearer $OMNIROUTE_API_KEY" http://proxy-ai.home/v1/models` → `200`. Routing (a)
  is confirmed end-to-end, no action needed.
- **MCP endpoint located from primary source** — fetched OmniRoute's
  `docs/frameworks/MCP-SERVER.md` at `release/v3.8.50` directly. Resolves the
  port ambiguity above: the MCP server runs on **port 20128** (dashboard/API
  port), paths `/api/mcp/stream` (streamable HTTP), `/api/mcp/sse`, and
  `/api/mcp/status`. It states: `/api/mcp/*` is in OmniRoute's `LOCAL_ONLY` authz
  tier (`src/server/authz/routeGuard.ts`) — loopback-only by default; a
  non-loopback client needs a key carrying the `manage` scope or the narrower
  `mcp:connect` scope (added v3.8.0), and the server's Settings must have
  `mcpEnabled` on with the matching `mcpTransport` selected. `omniroute_web_search`
  itself additionally needs `execute:search`. No separate "MCP key type" exists —
  same provider keys, different scopes.
- **Live probe result**: `curl http://proxy-ai.home:20128/api/mcp/status` returns
  `{"error":{"code":"AUTH_001","message":"Authentication required"}}` **identically
  with or without** the `Authorization: Bearer $OMNIROUTE_API_KEY` header — the
  existing model-routing key isn't recognized on this route at all, consistent
  with it lacking `mcp:connect`/`manage`/`execute:search` scope and/or
  `mcpEnabled` not yet being turned on in the dashboard. This is dashboard-side
  state (not in git, no session credentials available from this environment) —
  genuinely needs a human with dashboard access, not another probe.
- **Config prepared** to unblock as soon as that's done: added an
  `omniroute-search` entry to `~/.qwen/settings.json`'s `mcpServers` (backed up
  the prior file first as `settings.json.bak-wayfinder-<timestamp>`):
  ```json
  "omniroute-search": {
    "httpUrl": "http://proxy-ai.home:20128/api/mcp/stream",
    "headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
  }
  ```
  Deliberately a separate env var (`OMNIROUTE_SEARCH_KEY`), not reusing
  `OMNIROUTE_API_KEY`, so the search-scoped key stays distinct from the
  model-routing key — matches `docs/proxy-key-onboarding.md`'s per-workload
  key pattern.

## Resolution (2026-09-05, completed)

The dashboard steps above turned out to need a different diagnosis than
originally guessed — walked through live with a `oma_live_...` management
token and a rotating set of `sk-...` provider keys the user supplied:

- **`/api/providers` (management API) showed zero search providers at all**
  — not a misconfigured `searxng-search` entry, it simply didn't exist as a
  connection anymore (9 connections total, all LLM/chat providers). The
  CHANGELOG at the pinned ref was checked and shows `/v1/search` under active
  feature development (a `feat(search)` entry adding Firecrawl support), so
  this wasn't an OmniRoute-side removal of the search system — the row was
  just gone from this instance's own database (reason unconfirmed: update
  migration vs. prior manual removal).
- **Created it via the API**, not the dashboard UI — `POST /api/providers`
  turned out to accept the same generic connection schema used for LLM
  providers: `{"provider":"searxng-search","name":"searxng"}` (Zod-validated;
  an empty-body POST surfaced the required fields). Then
  `PATCH /api/providers/<id>` with `{"providerSpecificData":{"baseUrl":"http://search.home/search"}}`
  set the real URL, replacing the catalog default.
- **Verified end-to-end**: `POST /v1/search` with `provider: "searxng-search"`
  returned real results (5 hits, `search_cost_usd: 0`, `cached: false`,
  `response_time_ms: 4495`) — confirms `search.home`'s `extra_hosts` mapping
  in `docker-compose.yml` resolves correctly from inside the OmniRoute
  container and the whole chain (OmniRoute → SearXNG → results) works.
- **`/api/mcp/status` confirmed `scopesEnforced: false`** on this instance —
  the `mcp:connect`/`execute:search` scope requirement documented upstream
  isn't actually being enforced here, so any valid provider key connects.
  `mcpEnabled: true` already, transport `streamable-http`.
- **Key rotation caveat hit live**: the first `sk-...` key the user shared
  went from working to a flat 401 on *every* route (including `/v1/models`)
  partway through testing — consistent with it having been revoked/rotated
  server-side. A second key worked immediately. If this setup stops working
  later, check for exactly this before re-diagnosing the whole chain.
- **Final `~/.qwen/settings.json` `mcpServers` entry** (confirmed connected
  via `qwen mcp list` → `✓ omniroute-search: ... - Connected`):
  ```json
  "omniroute-search": {
    "httpUrl": "http://proxy-ai.home/api/mcp/stream",
    "headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
  }
  ```
  `OMNIROUTE_SEARCH_KEY` is exported in `~/.bashrc` alongside the existing
  `OMNIROUTE_API_KEY`, holding the second (working) `sk-...` key.

**Status: done.** `qwen` in WSL has a connected `omniroute-search` MCP server
backed by this stack's own SearXNG instance — no external search API, no
Alibaba DashScope dependency. Not yet exercised: an actual `qwen` chat turn
that triggers the `omniroute_web_search` tool call (only the MCP handshake
and the raw `/v1/search` call were verified directly).

## Open questions / unverified

- **`tools.webSearch.*` settings.json schema** — described on Qwen Code's
  web-search doc page but absent from the canonical settings-schema page; not
  independently confirmed (e.g. via `qwen --help` or source) — moot for this
  setup since the MCP path (above) is what's being wired in, not the
  DashScope-only built-in tool.
- **DashScope vs SearXNG data-residency/cost tradeoffs** — out of scope here,
  but worth noting the built-in `web_search` tool sends queries to Alibaba
  Cloud regardless of this stack being otherwise fully self-hosted.
- OmniRoute's own docs (already flagged in this repo's `README.md`) describe
  stealth/anti-detection and TLS-interception features elsewhere in its repo;
  none of that is exercised by anything in this note, but it's the same caveat
  `README.md` already carries forward from issue #31.