docs(research): add image-gen backend survey and omniroute/qwen websearch notes
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015zCwaWJQuKgXDUfRPBqDS7
This commit is contained in:
@@ -0,0 +1,249 @@
|
|||||||
|
# Research: adding local image generation to the stack
|
||||||
|
|
||||||
|
**Question:** What's the best way to add local image generation alongside
|
||||||
|
the existing Qwen3.8-27B / llama.cpp text stack, given a single AMD Radeon
|
||||||
|
R9700 (32GB VRAM, ROCm/gfx1201 — not CUDA), routed through the OmniRoute
|
||||||
|
gateway on the `ai-stack` Docker network?
|
||||||
|
|
||||||
|
**Answer, short version:** run **ComfyUI** (official AMD-blessed ROCm
|
||||||
|
Docker path exists, and OmniRoute already has a first-class `comfyui`
|
||||||
|
provider — no bespoke API wrapper needed) with **FLUX.1 [schnell]**
|
||||||
|
(Apache-2.0, 4-step, GGUF-quantizable) as the default model, falling back to
|
||||||
|
**SDXL** for anything schnell's distilled-step license/quality tradeoffs
|
||||||
|
don't suit. VRAM headroom against the current llama-server footprint is too
|
||||||
|
tight for both to be resident at once at any real image quality — plan for
|
||||||
|
**time-sliced use** (llama-server's existing lazytainer stop-on-idle pattern,
|
||||||
|
mirrored for the image-gen service, or a manual "stop one, start the other"
|
||||||
|
toggle), not concurrent operation.
|
||||||
|
|
||||||
|
## Current VRAM baseline (from this repo)
|
||||||
|
|
||||||
|
Per `docker-compose.yml` and `.env.example`, llama-server runs
|
||||||
|
`Qwen3.8-27B-UD-Q4_K_XL.gguf` (17.6 GB weights) at `--ctx-size 262144` with
|
||||||
|
`--cache-type-k q8_0 --cache-type-v q8_0`, landing at **~25.6 GB** total
|
||||||
|
(weights + q8_0 KV cache), leaving **~6 GB** free on the 32GB card — this
|
||||||
|
matches the math already recorded in
|
||||||
|
[`docs/research/qwen3.8-27b-quant.md`](qwen3.8-27b-quant.md). Per the auto-memory
|
||||||
|
note on this repo, real measured VRAM use has run closer to ~75% (~24 GB) in
|
||||||
|
practice versus the theoretical estimate, which doesn't change the
|
||||||
|
conclusion below but means the ~6 GB figure is closer to a ceiling than a
|
||||||
|
comfortable number.
|
||||||
|
|
||||||
|
**Implication:** 6 GB is not enough for any current-generation image model at
|
||||||
|
usable quality (see VRAM table below — even the smallest practical FLUX
|
||||||
|
quant wants ~7 GB alone, before ComfyUI's own runtime/VAE overhead). Running
|
||||||
|
image-gen *concurrently* with llama-server resident is not realistic on this
|
||||||
|
card. The two need to time-share the GPU, not split it.
|
||||||
|
|
||||||
|
## Backend evaluation (ROCm support, checked against primary sources)
|
||||||
|
|
||||||
|
### ComfyUI — recommended
|
||||||
|
|
||||||
|
- **Official AMD ROCm docs exist and are current.** AMD's own ROCm docs site
|
||||||
|
hosts a dedicated ComfyUI install guide with a prebuilt Docker image path
|
||||||
|
(recommended) or build-from-source, listing ROCm 7.2.0 and 7.1.0 as
|
||||||
|
supported versions, explicit `--device=/dev/kfd --device=/dev/dri
|
||||||
|
--group-add video` flags (same device-passthrough pattern this repo
|
||||||
|
already uses for llama-server), and template workflows including "SD3.5
|
||||||
|
Simple". Officially the guide only names AMD Instinct
|
||||||
|
MI355X/MI325X/MI300X (datacenter cards) as supported platforms.
|
||||||
|
Source: [ROCm docs — ComfyUI on ROCm installation](https://rocm.docs.amd.com/projects/comfyui/en/docs-26.04/install/comfyui-install.html).
|
||||||
|
- **The upstream ComfyUI README itself documents AMD support directly**,
|
||||||
|
including consumer cards: stable ROCm install via
|
||||||
|
`pip install torch torchvision torchaudio --index-url
|
||||||
|
https://download.pytorch.org/whl/rocm7.2`, plus an experimental Windows
|
||||||
|
build explicitly naming **RDNA 3 (RX 7000), RDNA 3.5 (Strix Halo), and
|
||||||
|
RDNA 4 (RX 9000 series)** — i.e. the same RDNA4 generation as the R9700 —
|
||||||
|
and `HSA_OVERRIDE_GFX_VERSION` workarounds for older/unlisted cards.
|
||||||
|
Source: [comfyanonymous/ComfyUI README](https://github.com/comfyanonymous/ComfyUI).
|
||||||
|
- **AMD has published a specific RDNA4/RX 9000 ComfyUI guide** (separate
|
||||||
|
from the Instinct-only install page above), confirming RDNA4 consumer
|
||||||
|
cards are an explicitly supported, first-party-documented target, not just
|
||||||
|
a community workaround.
|
||||||
|
Source: [ROCm blog — Getting Started with ComfyUI on AMD Radeon RX 9000 Series GPUs](https://rocm.blogs.amd.com/artificial-intelligence/comfyui-radeon-9000/README.html).
|
||||||
|
- **gfx1201 (R9700's arch) specifically has active community Docker images**:
|
||||||
|
`yurisasc/comfyui-rocm-rdna4` targets ROCm 7.1 + PyTorch 2.9.1 with
|
||||||
|
`HSA_OVERRIDE_GFX_VERSION=12.0.1` / `PYTORCH_ROCM_ARCH=gfx1201` baked in,
|
||||||
|
and there's a published community patch specifically for R9700 AI Pro +
|
||||||
|
ComfyUI video-gen speedups, evidence the card is being run today, not just
|
||||||
|
theoretically compatible.
|
||||||
|
Sources: [yurisasc/comfyui-rocm-rdna4](https://github.com/yurisasc/comfyui-rocm-rdna4),
|
||||||
|
[charlie12345/R9700AIProComfyUIPatch](https://github.com/charlie12345/R9700AIProComfyUIPatch).
|
||||||
|
- **Known gfx1201 caveat:** AMD's own TransformerEngine repo has an open
|
||||||
|
issue confirming gfx1201 is missing from the FP8 architecture table, so
|
||||||
|
FP8 kernels silently fall back to FP32 with ~50% throughput loss
|
||||||
|
(18-22 vs. 35-40 tok/s in the reporter's LLM benchmark) — not a
|
||||||
|
correctness blocker, but relevant if planning to use FP8-quantized image
|
||||||
|
models expecting native FP8 speed on this card; GGUF/Q-quants (see below)
|
||||||
|
avoid this path entirely since they dequantize to bf16/fp16, not fp8.
|
||||||
|
Source: [ROCm/TransformerEngine issue #520](https://github.com/ROCm/TransformerEngine/issues/520).
|
||||||
|
- **Actively maintained community Docker packaging** beyond AMD's own image:
|
||||||
|
`YanWenKun/ComfyUI-Docker` ships parallel `rocm` (PyTorch-build-based,
|
||||||
|
faster releases) and `rocm7` (AMD-build-based, more comprehensive)
|
||||||
|
variants, both targeting ROCm 7, with ~1000 commits of ongoing history —
|
||||||
|
a viable alternative to the official AMD image if it lags behind ComfyUI
|
||||||
|
releases.
|
||||||
|
Source: [YanWenKun/ComfyUI-Docker](https://github.com/YanWenKun/ComfyUI-Docker).
|
||||||
|
|
||||||
|
### AUTOMATIC1111 / Forge — usable but a step down for this hardware
|
||||||
|
|
||||||
|
- ROCm support for A1111/Forge is real but community-patched, not
|
||||||
|
first-party. The upstream `lllyasviel/stable-diffusion-webui-forge` repo's
|
||||||
|
own discussion thread on AMD support points users to
|
||||||
|
`lshqqytiger/stable-diffusion-webui-amdgpu-forge`, a community fork
|
||||||
|
specifically maintained for AMD, "regarded as the go-to version" for
|
||||||
|
running FLUX-era models on AMD — i.e. the *mainline* Forge repo does not
|
||||||
|
claim ROCm support itself; you're expected to run a fork.
|
||||||
|
Source: [lllyasviel/stable-diffusion-webui-forge discussion #67](https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/67).
|
||||||
|
- No first-party AMD vendor documentation (unlike ComfyUI's AMD-authored
|
||||||
|
ROCm/RDNA4 blog posts above) was found for A1111/Forge specifically.
|
||||||
|
Given ComfyUI already has an AMD-blessed path plus a first-class OmniRoute
|
||||||
|
provider (below), there's no reason to take on a community fork's
|
||||||
|
maintenance risk instead.
|
||||||
|
|
||||||
|
### InvokeAI — usable but weaker AMD story for a new-generation card
|
||||||
|
|
||||||
|
- InvokeAI documents ROCm support but flags it as second-tier: "AMD GPUs
|
||||||
|
are only supported on Linux," and "support for newer AMD GPUs is spotty
|
||||||
|
... you may experience garbled images, black images, or long startup
|
||||||
|
delays." Its own install docs reference ROCm 5.4.2-era wheels, notably
|
||||||
|
older than the ROCm 7.x this stack's llama-server image already runs on
|
||||||
|
gfx1201.
|
||||||
|
Source: [InvokeAI installation docs (mauwii mirror)](https://mauwii.github.io/InvokeAI/installation/030_INSTALL_CUDA_AND_ROCM/).
|
||||||
|
- No OpenAI-compatible-images angle either — same drawback as A1111/Forge.
|
||||||
|
Not recommended as primary given ComfyUI's stronger, more current AMD
|
||||||
|
documentation trail.
|
||||||
|
|
||||||
|
## Model choice: FLUX.1 [schnell] vs FLUX.1 [dev] vs SDXL vs SD3.5
|
||||||
|
|
||||||
|
| Model | License | Params | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **FLUX.1 [schnell]** | **Apache-2.0** — fully open, no commercial restriction | 12B | Distilled for 1-4 step inference (fast); Black Forest Labs' own model card states this license directly |
|
||||||
|
| FLUX.1 [dev] | [FLUX.1-dev Non-Commercial License](https://github.com/black-forest-labs/flux/blob/main/model_licenses/LICENSE-FLUX1-dev) | 12B | Non-commercial for the *model/weights*; generated *outputs* are explicitly usable commercially per BFL's license page. Higher quality than schnell (more steps, non-distilled) but the weights themselves can't be redistributed/used commercially |
|
||||||
|
| SDXL | CreativeML OpenRAIL++ (permissive, commercial-friendly) | ~3.5B | Older (2023), lower fidelity than FLUX/SD3.5 by current standards, but lowest VRAM footprint and best long-standing tooling maturity |
|
||||||
|
| SD3.5 (Large/Medium) | Stability AI Community License — free commercial use under $1M annual revenue, else enterprise license required | 8B / 2.5B | Free for this repo's non-commercial homelab use regardless; template already listed in AMD's own ComfyUI-ROCm doc ("SD3.5 Simple") as a first-party example workflow |
|
||||||
|
|
||||||
|
Sources: [black-forest-labs/flux model cards](https://github.com/black-forest-labs/flux/blob/main/model_cards/FLUX.1-dev.md),
|
||||||
|
[black-forest-labs/FLUX.1-schnell on Hugging Face](https://huggingface.co/black-forest-labs/FLUX.1-schnell)
|
||||||
|
(license: apache-2.0), [FLUX.1-dev LICENSE.md](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md),
|
||||||
|
[Stability AI — Introducing Stable Diffusion 3.5](https://stability.ai/news-updates/introducing-stable-diffusion-3-5),
|
||||||
|
[stabilityai/stable-diffusion-3.5-large LICENSE.md](https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/LICENSE.md).
|
||||||
|
|
||||||
|
**Recommendation: FLUX.1 [schnell].** For a private homelab, license
|
||||||
|
enforcement isn't the deciding factor by itself, but schnell's Apache-2.0
|
||||||
|
status removes any future ambiguity if outputs or the setup are ever shared
|
||||||
|
or repurposed, and its whole design point — good quality in 1-4 sampling
|
||||||
|
steps — directly addresses the VRAM/time-slicing constraint below (less
|
||||||
|
GPU-resident time per image than a 20-50 step dev/SDXL/SD3.5 run).
|
||||||
|
Quantized via `city96/ComfyUI-GGUF` (an actively-referenced, community-
|
||||||
|
trusted quantization node — its GGUF Q-quants dequantize to bf16/fp16 at
|
||||||
|
runtime, sidestepping the gfx1201 FP8 dequant bug above entirely), FLUX fits
|
||||||
|
in a fraction of its fp16 footprint:
|
||||||
|
|
||||||
|
| Precision | Approx. VRAM (model only) |
|
||||||
|
|---|---|
|
||||||
|
| fp16 (baseline) | ~24 GB |
|
||||||
|
| fp8 | ~12 GB |
|
||||||
|
| GGUF Q5_K_S | ~12-15 GB (practical quality floor) |
|
||||||
|
| GGUF Q4_K_S | ~7 GB (quality starts degrading on hands/text below Q4) |
|
||||||
|
|
||||||
|
Source: aggregated VRAM figures from GGUF-quantization write-ups referencing
|
||||||
|
city96's FLUX GGUF conversions — treat as secondary/community sourced
|
||||||
|
(no single BFL-published VRAM table was found), consistent across multiple
|
||||||
|
independent sources.
|
||||||
|
[city96/ComfyUI-GGUF README](https://github.com/city96/ComfyUI-GGUF/blob/main/README.md),
|
||||||
|
[city96/FLUX.1-dev-gguf model card](https://huggingface.co/city96/FLUX.1-dev-gguf).
|
||||||
|
|
||||||
|
**Fallback pick: SDXL.** If schnell's distilled quality ceiling proves too
|
||||||
|
low for some use case, SDXL is the safer second choice over FLUX.1 [dev] or
|
||||||
|
SD3.5 specifically *because* of this card's tight headroom: it's the
|
||||||
|
smallest of the four by a wide margin, has the longest production track
|
||||||
|
record on ROCm of any of these models, and its OpenRAIL++ license carries no
|
||||||
|
revenue-threshold clause to track (unlike SD3.5's Community License) or
|
||||||
|
non-commercial weight restriction (unlike FLUX.1 [dev]).
|
||||||
|
|
||||||
|
## OpenAI-compatible API / OmniRoute integration
|
||||||
|
|
||||||
|
This is the best news in this research: **OmniRoute already ships a
|
||||||
|
first-class, built-in `comfyui` provider** — not a generic "point it at an
|
||||||
|
OpenAI base URL and hope" integration. Its provider reference documents it
|
||||||
|
explicitly: *"No API key required. Configure the local ComfyUI base URL
|
||||||
|
(default: http://localhost:8188)."* OmniRoute's own image-routing feature
|
||||||
|
set (`/v1/images/generations`, `/v1/images/edits`, `/v1/images/variations`,
|
||||||
|
with automatic provider fallback) is designed for exactly this pattern:
|
||||||
|
register ComfyUI as a backend, then any client already calling OmniRoute's
|
||||||
|
OpenAI-compatible images endpoints reaches it with no extra shim.
|
||||||
|
Source: [diegosouzapw/OmniRoute PROVIDER_REFERENCE.md](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/PROVIDER_REFERENCE.md),
|
||||||
|
[diegosouzapw/OmniRoute repo description](https://github.com/diegosouzapw/OmniRoute).
|
||||||
|
|
||||||
|
This means **ComfyUI does not need an extra OpenAI-API wrapper project** —
|
||||||
|
the wrapper projects found during this research
|
||||||
|
([`ComfyUI-OpenAI-Compatible-API`](https://github.com/yeeyou/ComfyUI-OpenAI-Compatible-API))
|
||||||
|
turned out to be a ComfyUI *custom node* for calling *outbound* to LLM APIs
|
||||||
|
from within a workflow (the reverse direction), not something this stack
|
||||||
|
needs — OmniRoute's own native ComfyUI provider is the actual integration
|
||||||
|
point, one layer up.
|
||||||
|
|
||||||
|
**Confidence note:** the provider-reference detail above was fetched via an
|
||||||
|
automated summarizer against the raw doc rather than manually re-verified
|
||||||
|
line-by-line; re-check `PROVIDER_REFERENCE.md`'s `comfyui` entry directly
|
||||||
|
before wiring this up, in case ComfyUI's own `/prompt` API (a
|
||||||
|
workflow-graph-shaped API, not a simple text-prompt-in/image-out call) needs
|
||||||
|
a specific default workflow JSON configured on the OmniRoute side to produce
|
||||||
|
a plain text-to-image call.
|
||||||
|
|
||||||
|
## Integration sketch (not a full compose — see caveats above)
|
||||||
|
|
||||||
|
- New service in `docker-compose.yml`, e.g. `comfyui`, image
|
||||||
|
`rocm/comfyui-rocm` (or `yurisasc/comfyui-rocm-rdna4` for a gfx1201-tuned
|
||||||
|
build) or built from AMD's own ROCm ComfyUI Dockerfile, same
|
||||||
|
`/dev/kfd` + `/dev/dri` + `group_add: [video, render]` device-passthrough
|
||||||
|
block already used for `llama-server`, joined to the same `ai-stack`
|
||||||
|
network so `omniroute` can reach it as `http://comfyui:8188` — no host
|
||||||
|
port needed (matches the existing llama-server pattern of no published
|
||||||
|
port, gateway-only access).
|
||||||
|
- Register it in OmniRoute's dashboard as a `comfyui` provider pointing at
|
||||||
|
that internal URL, same manual-registration pattern already used for
|
||||||
|
llama-server and searxng-search per `docs/proxy-key-onboarding.md`.
|
||||||
|
- **VRAM contention is the real design problem, not networking.** Given the
|
||||||
|
~6 GB headroom, the two services can't both sit GPU-resident.
|
||||||
|
Two workable patterns, in order of how well they fit what's already in
|
||||||
|
this repo:
|
||||||
|
1. **Mirror the existing lazytainer stop-on-idle pattern** already applied
|
||||||
|
to `llama-server` (`docker-compose.yml`'s `lazytainer.group.*` labels) —
|
||||||
|
add an equivalent idle-timeout group for `comfyui`, and rely on the two
|
||||||
|
services naturally not being hit at the same time for a single-user
|
||||||
|
homelab. This doesn't *guarantee* mutual exclusion (both could still be
|
||||||
|
woken concurrently and both try to fit in 6 GB free), so it's a
|
||||||
|
reasonable-effort fit, not a hard guarantee.
|
||||||
|
2. **Explicit mutual exclusion**: a small script/compose profile that
|
||||||
|
stops `llama-server` before starting `comfyui` (and vice versa) rather
|
||||||
|
than relying on lazytainer's independent idle timers — worth doing if
|
||||||
|
the reasonable-effort version above causes a visible OOM in practice.
|
||||||
|
Either way, this is a "pick one, then the other" story, not "run both."
|
||||||
|
- Given FLUX.1 [schnell]'s 1-4 step design, a cold-start-and-generate cycle
|
||||||
|
(wake ComfyUI from lazytainer sleep, generate, let it idle back down) is
|
||||||
|
a reasonably good fit for occasional image requests through the same
|
||||||
|
gateway that already does this for llama-server.
|
||||||
|
|
||||||
|
## Sources consulted
|
||||||
|
|
||||||
|
- [ROCm docs — ComfyUI on ROCm installation](https://rocm.docs.amd.com/projects/comfyui/en/docs-26.04/install/comfyui-install.html)
|
||||||
|
- [ROCm blog — ComfyUI on AMD Radeon RX 9000 Series (RDNA4)](https://rocm.blogs.amd.com/artificial-intelligence/comfyui-radeon-9000/README.html)
|
||||||
|
- [comfyanonymous/ComfyUI README](https://github.com/comfyanonymous/ComfyUI)
|
||||||
|
- [YanWenKun/ComfyUI-Docker](https://github.com/YanWenKun/ComfyUI-Docker)
|
||||||
|
- [yurisasc/comfyui-rocm-rdna4](https://github.com/yurisasc/comfyui-rocm-rdna4)
|
||||||
|
- [charlie12345/R9700AIProComfyUIPatch](https://github.com/charlie12345/R9700AIProComfyUIPatch)
|
||||||
|
- [ROCm/TransformerEngine issue #520 (gfx1201 FP8 fallback)](https://github.com/ROCm/TransformerEngine/issues/520)
|
||||||
|
- [lllyasviel/stable-diffusion-webui-forge discussion #67 (AMD support)](https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/67)
|
||||||
|
- [InvokeAI CUDA/ROCm install docs](https://mauwii.github.io/InvokeAI/installation/030_INSTALL_CUDA_AND_ROCM/)
|
||||||
|
- [black-forest-labs/flux GitHub repo + model cards](https://github.com/black-forest-labs/flux)
|
||||||
|
- [black-forest-labs/FLUX.1-schnell (Hugging Face, Apache-2.0)](https://huggingface.co/black-forest-labs/FLUX.1-schnell)
|
||||||
|
- [black-forest-labs/FLUX.1-dev LICENSE.md](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md)
|
||||||
|
- [Stability AI — Introducing Stable Diffusion 3.5](https://stability.ai/news-updates/introducing-stable-diffusion-3-5)
|
||||||
|
- [stabilityai/stable-diffusion-3.5-large LICENSE.md](https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/LICENSE.md)
|
||||||
|
- [city96/ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF)
|
||||||
|
- [city96/FLUX.1-dev-gguf](https://huggingface.co/city96/FLUX.1-dev-gguf)
|
||||||
|
- [diegosouzapw/OmniRoute](https://github.com/diegosouzapw/OmniRoute) and its `PROVIDER_REFERENCE.md`
|
||||||
|
- [yeeyou/ComfyUI-OpenAI-Compatible-API](https://github.com/yeeyou/ComfyUI-OpenAI-Compatible-API) (checked and ruled out — wrong direction)
|
||||||
|
- This repo: `docker-compose.yml`, `.env.example`, `docs/research/qwen3.8-27b-quant.md`
|
||||||
@@ -0,0 +1,330 @@
|
|||||||
|
# OmniRoute + Qwen Code CLI web search — setup research
|
||||||
|
|
||||||
|
Investigates how to (a) confirm/complete OmniRoute's routing to this stack's local
|
||||||
|
Qwen model, and (b) enable Qwen Code CLI's web-search tool, for a user running
|
||||||
|
`qwen` from WSL against this repo's docker-compose stack.
|
||||||
|
|
||||||
|
## What's already configured (verified live in WSL, 2026-09-05)
|
||||||
|
|
||||||
|
Checked via `wsl.exe -- bash -lc '...'` against `~/.qwen/`:
|
||||||
|
|
||||||
|
- **qwen-code CLI is installed**: `which qwen` → `/home/haylan/.local/bin/qwen`, `qwen --version` → `0.23.0`.
|
||||||
|
- **`~/.qwen/settings.json` already points at this stack's OmniRoute gateway**, in the exact shape OmniRoute's own `setup-qwen` command produces (see below):
|
||||||
|
```json
|
||||||
|
"modelProviders": {
|
||||||
|
"openai": [
|
||||||
|
{
|
||||||
|
"id": "qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf",
|
||||||
|
"name": "qwen3.8-27b-local",
|
||||||
|
"envKey": "OMNIROUTE_API_KEY",
|
||||||
|
"baseUrl": "http://proxy-ai.home/v1",
|
||||||
|
"generationConfig": { "contextWindowSize": 131072 }
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"security": { "auth": { "selectedType": "openai" } },
|
||||||
|
"model": {
|
||||||
|
"name": "qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf",
|
||||||
|
"baseUrl": "http://proxy-ai.home/v1"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
This targets `http://proxy-ai.home/v1` (this repo's OmniRoute gateway hostname per `docs/network-access.md`), reads the API key from the `OMNIROUTE_API_KEY` env var, and matches `docs/coding-cli-setup.md`'s convention of naming the registered provider `qwen3.8-27b-local`. Two backup files (`settings.json.bak-cbm-*`, `settings.json.save`) show earlier iterations of the same config — this was set up deliberately, not a stray default.
|
||||||
|
- **Not a gap — verified correct**: `contextWindowSize: 131072` matches `LLAMA_CTX_SIZE / LLAMA_PARALLEL` (`262144 / 2`), not half of it. `docker-compose.yml` (lines 21–22) runs llama-server with `--ctx-size ${LLAMA_CTX_SIZE:-262144} --parallel ${LLAMA_PARALLEL:-2}`, and `.env.example` (line 39) spells out that each of the two concurrent slots gets `LLAMA_CTX_SIZE / LLAMA_PARALLEL` tokens — i.e. 131072 per slot, matching commit `23e90fe` ("cap concurrent slots at 2"). So `~/.qwen/settings.json`'s value is correctly sized to what one slot actually offers; no fix needed here.
|
||||||
|
- **Unverified**: whether `OMNIROUTE_API_KEY` is actually set in the WSL environment or in a `~/.qwen/.env` file — `env | grep -i qwen` in the same session showed no `OMNIROUTE_API_KEY` in the *current* shell (only `PATH` entries matched `qwen`), and `~/.qwen/.env` wasn't checked (missed in the executed probe — see Open questions). If it's unset, `qwen` calls will fail auth against OmniRoute regardless of the `web_search` setup below.
|
||||||
|
- **No web-search config exists yet**: `env | grep -i tavily` and `env | grep -i search` both returned nothing; `settings.json` has no `tools.webSearch` key and no `mcpServers` entry for Tavily/Bailian/GLM search or for OmniRoute's own MCP server (it does have an unrelated `mcpServers.codebase-memory-mcp` stdio entry for this repo's own codebase-memory tool). `grep -ril "tavily\|websearch\|web_search\|web-search" ~/.qwen` matched only unrelated project chat-log files (from an unrelated `shopware-420-seeds` project), not any config.
|
||||||
|
- **Conclusion**: model routing (a) is already done. Web search (b) is not configured at all — no API key, no MCP server, no built-in-tool setting.
|
||||||
|
|
||||||
|
## (a) OmniRoute → local Qwen model routing
|
||||||
|
|
||||||
|
Sources: this repo's `docker-compose.yml` (lines 63–133) and `.env.example`
|
||||||
|
(lines 54–90); `README.md` §"AI gateway (OmniRoute)"; `docs/coding-cli-setup.md`;
|
||||||
|
OmniRoute's own docs at `github.com/mckazzy/OmniRoute-run-qwen`, ref
|
||||||
|
`release/v3.8.50`.
|
||||||
|
|
||||||
|
**Current repo state**: `docker-compose.yml`'s `omniroute` service comment (lines 63–67)
|
||||||
|
states routing is registered "once through the dashboard or `POST /api/providers`
|
||||||
|
after first boot, not checked into this repo." `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true`
|
||||||
|
and `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS=true` are already set (lines 84–88) so the
|
||||||
|
dashboard/API will accept `llama-server`'s container-internal address instead of
|
||||||
|
rejecting it as a private URL.
|
||||||
|
|
||||||
|
**Confirmed against OmniRoute's own reference docs**
|
||||||
|
(`docs/reference/ENVIRONMENT.md` at the pinned ref):
|
||||||
|
- `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` — default `false`; the doc says it is
|
||||||
|
**"REQUIRED for self-hosted providers"** (it names LM Studio, Ollama, vLLM,
|
||||||
|
Llamafile, Triton, SearXNG). Confirms the repo's own comment is correct and necessary.
|
||||||
|
- `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS` — default `true` ("local-first"); `false`
|
||||||
|
would block localhost/LAN/private ranges outright (cloud-metadata IPs stay
|
||||||
|
blocked either way).
|
||||||
|
- `OMNIROUTE_WS_BRIDGE_SECRET` — "REQUIRED in production — when unset, all WS
|
||||||
|
bridge requests are rejected," generated via `openssl rand -base64 32` — matches
|
||||||
|
this repo's comment (lines 89–93) and `scripts/update.sh` autofill.
|
||||||
|
|
||||||
|
`docs/reference/PROVIDER_REFERENCE.md` (same ref) lists **`llama-cpp`** as a
|
||||||
|
built-in "Local, self-hosted" provider ID:
|
||||||
|
|
||||||
|
> "Configure the OpenAI-compatible base URL (default: `http://127.0.0.1:8080/v1`)"
|
||||||
|
|
||||||
|
This is a good match for this stack's `llama-server` container, which exposes
|
||||||
|
port 8080 only on the internal `ai-stack` Docker network (`docker-compose.yml`
|
||||||
|
lines 30–33, "No published host port"). Inside that network the service is
|
||||||
|
reachable by its Compose service name, so the base URL to register should be
|
||||||
|
`http://llama-server:8080/v1`, not `127.0.0.1` (127.0.0.1 inside the OmniRoute
|
||||||
|
container would mean OmniRoute itself, not llama-server — they're different
|
||||||
|
containers on the same bridge network).
|
||||||
|
|
||||||
|
**Concrete steps** (dashboard, matching `docs/proxy-key-onboarding.md`'s
|
||||||
|
existing pattern for reaching the dashboard):
|
||||||
|
|
||||||
|
1. Reach the dashboard: from the R9700 box, `docker inspect -f
|
||||||
|
'{{.NetworkSettings.Networks.ai_stack.IPAddress}}' omniroute`, then browse
|
||||||
|
`http://<that-ip>:20128`; from elsewhere, SSH-tunnel
|
||||||
|
`ssh -L 20128:<container-ip>:20128 <host>` then browse `localhost:20128`.
|
||||||
|
2. Providers → Add provider → **llama.cpp** (`llama-cpp` provider ID per
|
||||||
|
`PROVIDER_REFERENCE.md`).
|
||||||
|
3. Set base URL to `http://llama-server:8080/v1` (the Compose service name — both
|
||||||
|
containers share the `ai-stack` network per `docker-compose.yml`'s `networks:
|
||||||
|
[ai-stack]` on both services). No API key needed (llama-server's endpoint is
|
||||||
|
unauthenticated internally, per `docs/network-access.md`).
|
||||||
|
4. Register the model under that provider using the naming this repo already
|
||||||
|
assumes downstream (`qwen3.8-27b-local`, per `docs/coding-cli-setup.md` line 8)
|
||||||
|
— pick a model ID/name here and keep it consistent everywhere a CLI config
|
||||||
|
references it (`~/.qwen/settings.json`'s existing entry already assumes this name).
|
||||||
|
5. Mint or reuse a virtual API key for the `qwen-code-cli` workload per
|
||||||
|
`docs/proxy-key-onboarding.md` (label `qwen-code-cli`), and confirm it's the
|
||||||
|
value behind `OMNIROUTE_API_KEY` in the WSL environment (or `~/.qwen/.env` —
|
||||||
|
see Open questions) that `~/.qwen/settings.json`'s `envKey` references.
|
||||||
|
|
||||||
|
**OmniRoute's own automation for this exact CLI** — `docs/guides/CLI-INTEGRATIONS.md`
|
||||||
|
at the pinned ref documents a dedicated `omniroute setup-qwen` command:
|
||||||
|
|
||||||
|
> `omniroute setup-qwen --model qwen/qwen3.8-max-preview` — writes
|
||||||
|
> `~/.qwen/settings.json` (V4 `modelProviders.openai` array) and stores
|
||||||
|
> `OMNIROUTE_API_KEY` in `~/.qwen/.env`; supports `--yes` (non-interactive),
|
||||||
|
> `--config-path` / `--env-path` (custom locations), and works in local or remote mode.
|
||||||
|
|
||||||
|
The `~/.qwen/settings.json` found on this machine has exactly the V4
|
||||||
|
`modelProviders.openai` shape this command produces, and the two `.bak`/`.save`
|
||||||
|
files back that up — this was very likely already run once, pointed at whichever
|
||||||
|
model ID was registered in the dashboard at the time (the `id` field embeds the
|
||||||
|
GGUF filename, `qwen3.8-27b-local//models/Qwen3.8-27B-UD-Q4_K_XL.gguf`, matching
|
||||||
|
`.env.example`'s `LLAMA_MODEL_FILE`). Re-running it after registering/renaming
|
||||||
|
the provider in step 2–4 above is the fastest way to refresh this file if the
|
||||||
|
registered model ID ever changes (`contextWindowSize: 131072` itself is already
|
||||||
|
correct — see note above on `--parallel`).
|
||||||
|
|
||||||
|
## (b) Qwen Code CLI web search
|
||||||
|
|
||||||
|
Sources: `qwenlm.github.io/qwen-code-docs/en/developers/tools/web-search/`,
|
||||||
|
`.../en/developers/tools/mcp-server/`, `.../en/users/configuration/settings/`;
|
||||||
|
OmniRoute's `docs/frameworks/MCP-SERVER.md` and `docs/reference/PROVIDER_REFERENCE.md`
|
||||||
|
at `release/v3.8.50`.
|
||||||
|
|
||||||
|
**Qwen Code's web-search docs page states plainly**: the *original* built-in
|
||||||
|
`web_search` tool ("Tavily/Google/GLM/DashScope multi-provider") **"and its
|
||||||
|
configuration were removed."** Current options, per that same page:
|
||||||
|
|
||||||
|
1. **New built-in `web_search` tool** — DashScope-only now, not multi-provider.
|
||||||
|
Needs `tools.webSearch.enabled: true` and `tools.webSearch.model` (e.g.
|
||||||
|
`"qwen3.6-plus"`) in `settings.json`, or equivalent env vars if `settings.json`
|
||||||
|
can't be edited; requires a `DASHSCOPE_API_KEY` (Alibaba Cloud). It "issues a
|
||||||
|
self-contained search request to a small auxiliary model with DashScope's
|
||||||
|
server-side `web_search` (and `web_extractor`) tools, and returns the
|
||||||
|
narrated findings plus source URLs" — i.e. it calls out to Alibaba's cloud,
|
||||||
|
not this stack's local model or SearXNG.
|
||||||
|
- **Caveat**: `users/configuration/settings/` (the canonical settings-schema
|
||||||
|
page) does **not** list `tools.webSearch` anywhere among its documented
|
||||||
|
`tools.*` keys — only `tools.sandbox`, `tools.shell`, `tools.core`,
|
||||||
|
`tools.exclude`, `tools.disabled`. This key may be genuinely undocumented
|
||||||
|
there, or newer than that page's last update. Treat `tools.webSearch` as
|
||||||
|
unconfirmed against the settings schema itself — verify with `qwen --help`
|
||||||
|
or by testing once a `DASHSCOPE_API_KEY` is available (see Open questions).
|
||||||
|
2. **MCP-based search** — three named services: Alibaba Cloud Bailian WebSearch,
|
||||||
|
Tavily WebSearch, GLM WebSearch Prime — each added as an `mcpServers` entry
|
||||||
|
in `settings.json`. Confirmed schema from `developers/tools/mcp-server/`:
|
||||||
|
HTTP/SSE servers use `httpUrl` (or `url` for SSE) plus an optional `headers`
|
||||||
|
object, e.g.:
|
||||||
|
```json
|
||||||
|
{ "mcpServers": { "tavily": {
|
||||||
|
"httpUrl": "https://mcp.tavily.com/mcp/?tavilyApiKey=${TAVILY_API_KEY}"
|
||||||
|
} } }
|
||||||
|
```
|
||||||
|
(stdio servers instead use `command`/`args`/`env`/`cwd`, as the existing
|
||||||
|
`codebase-memory-mcp` entry in this machine's `~/.qwen/settings.json` does.)
|
||||||
|
|
||||||
|
**Neither of Qwen Code's own two paths uses this stack's existing SearXNG
|
||||||
|
integration.** But OmniRoute — already in front of this stack's model — has its
|
||||||
|
own MCP server with a **built-in multi-provider web-search tool**, and this
|
||||||
|
repo already wires SearXNG through OmniRoute (`README.md` §"Web search":
|
||||||
|
"The gateway also fronts SearXNG-backed web search"; `.env.example`'s
|
||||||
|
`SEARXNG_LAN_IP` / `search.home` extra_hosts entry in `docker-compose.yml`
|
||||||
|
lines 106–109). OmniRoute's `docs/frameworks/MCP-SERVER.md` (pinned ref):
|
||||||
|
|
||||||
|
> "Web search through OmniRoute search gateway
|
||||||
|
> (Serper/Brave/Perplexity/Exa/Tavily/Google PSE/Linkup/SearchAPI/SearXNG) with
|
||||||
|
> failover" — exposed as an `omniroute_web_search` tool requiring the
|
||||||
|
> `execute:search` scope.
|
||||||
|
|
||||||
|
And `docs/reference/PROVIDER_REFERENCE.md` lists `searxng-search` as one of its
|
||||||
|
12 built-in search-provider IDs: **"API key is optional. Set your SearXNG base
|
||||||
|
URL. Some instances may require a bearer token for access."** — meaning
|
||||||
|
SearXNG can be registered as a search provider in the OmniRoute dashboard the
|
||||||
|
same way `llama-cpp` is registered as a model provider, no separate API key
|
||||||
|
needed for a self-hosted SearXNG instance.
|
||||||
|
|
||||||
|
**This means the path that reuses what's already deployed in this stack (SearXNG,
|
||||||
|
already reachable from OmniRoute via `search.home`) is: connect qwen-code to
|
||||||
|
OmniRoute's MCP server, not to Tavily/DashScope/GLM directly.** Concrete steps:
|
||||||
|
|
||||||
|
1. In the OmniRoute dashboard, register SearXNG as a search provider
|
||||||
|
(`searxng-search`), pointing at `http://search.home` (already resolvable
|
||||||
|
inside the OmniRoute container via the `extra_hosts` entry in
|
||||||
|
`docker-compose.yml`). This may already be done — `README.md` implies the
|
||||||
|
gateway already fronts SearXNG-backed search, but confirm live in the
|
||||||
|
dashboard since, per the same `docker-compose.yml` comment (lines 63–67),
|
||||||
|
provider registration isn't checked into this repo.
|
||||||
|
2. Mint an API key scoped for MCP search use — OmniRoute's `MCP-SERVER.md`
|
||||||
|
names `execute:search` (to actually call the search tool) and `mcp:connect`
|
||||||
|
(narrow, MCP-connect-only) as the relevant scopes; `manage`/`admin` also work
|
||||||
|
but are broader than needed.
|
||||||
|
3. Add an `mcpServers` entry to `~/.qwen/settings.json` pointing at OmniRoute's
|
||||||
|
MCP endpoint, using the same `httpUrl`/`headers` shape Qwen Code already
|
||||||
|
supports for Tavily:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"omniroute-search": {
|
||||||
|
"httpUrl": "http://proxy-ai.home/api/mcp/stream",
|
||||||
|
"headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
(`proxy-ai.home` matches the hostname the model-provider entry already uses
|
||||||
|
in this same file; swap in whatever host:port actually fronts OmniRoute's API
|
||||||
|
port from WSL — `docs/network-access.md` says `proxy-ai.home` points at
|
||||||
|
`${OMNIROUTE_PORT:-4000}`, the *API* port, and `docker-compose.yml`/`.env.example`
|
||||||
|
separately track `OMNIROUTE_API_PORT` (default `20129`) as the
|
||||||
|
container-internal port — confirm which one NPM actually proxies to before
|
||||||
|
trusting the `/api/mcp/stream` path resolves through `proxy-ai.home` unchanged;
|
||||||
|
this wasn't independently verified against a live instance, see Open questions.)
|
||||||
|
4. Set `OMNIROUTE_SEARCH_KEY` in the WSL shell profile (or in `~/.qwen/.env`,
|
||||||
|
consistent with how `setup-qwen` already stores `OMNIROUTE_API_KEY` there).
|
||||||
|
5. Restart `qwen`; the model should now see an MCP tool for web search backed by
|
||||||
|
this stack's own SearXNG, routed and rate-limited the same way its LLM calls
|
||||||
|
already are.
|
||||||
|
|
||||||
|
If instead the goal is simply "get *any* web search working fastest, reuse
|
||||||
|
nothing," the plain Tavily-MCP or DashScope built-in-tool paths above are
|
||||||
|
simpler (one API key, no dashboard provider registration) — but they bypass
|
||||||
|
this stack's OmniRoute/SearXNG setup entirely and send queries to an external
|
||||||
|
paid API instead.
|
||||||
|
|
||||||
|
## Follow-up verification (2026-09-05, live checks)
|
||||||
|
|
||||||
|
- **`OMNIROUTE_API_KEY` — confirmed set and working.** It's exported from
|
||||||
|
`~/.bashrc` (line 133), *not* `~/.qwen/.env` — invisible to a non-interactive
|
||||||
|
`bash -lc` probe because `.bashrc`'s standard top-of-file guard
|
||||||
|
(`case $- in *i*) ;; *) return;; esac`) skips the rest of the file for
|
||||||
|
non-interactive shells; a real interactive shell (`bash -ic`, or `wsl` +
|
||||||
|
`qwen` as actually run) sources it fine. Verified: `curl -H "Authorization:
|
||||||
|
Bearer $OMNIROUTE_API_KEY" http://proxy-ai.home/v1/models` → `200`. Routing (a)
|
||||||
|
is confirmed end-to-end, no action needed.
|
||||||
|
- **MCP endpoint located from primary source** — fetched OmniRoute's
|
||||||
|
`docs/frameworks/MCP-SERVER.md` at `release/v3.8.50` directly. Resolves the
|
||||||
|
port ambiguity above: the MCP server runs on **port 20128** (dashboard/API
|
||||||
|
port), paths `/api/mcp/stream` (streamable HTTP), `/api/mcp/sse`, and
|
||||||
|
`/api/mcp/status`. It states: `/api/mcp/*` is in OmniRoute's `LOCAL_ONLY` authz
|
||||||
|
tier (`src/server/authz/routeGuard.ts`) — loopback-only by default; a
|
||||||
|
non-loopback client needs a key carrying the `manage` scope or the narrower
|
||||||
|
`mcp:connect` scope (added v3.8.0), and the server's Settings must have
|
||||||
|
`mcpEnabled` on with the matching `mcpTransport` selected. `omniroute_web_search`
|
||||||
|
itself additionally needs `execute:search`. No separate "MCP key type" exists —
|
||||||
|
same provider keys, different scopes.
|
||||||
|
- **Live probe result**: `curl http://proxy-ai.home:20128/api/mcp/status` returns
|
||||||
|
`{"error":{"code":"AUTH_001","message":"Authentication required"}}` **identically
|
||||||
|
with or without** the `Authorization: Bearer $OMNIROUTE_API_KEY` header — the
|
||||||
|
existing model-routing key isn't recognized on this route at all, consistent
|
||||||
|
with it lacking `mcp:connect`/`manage`/`execute:search` scope and/or
|
||||||
|
`mcpEnabled` not yet being turned on in the dashboard. This is dashboard-side
|
||||||
|
state (not in git, no session credentials available from this environment) —
|
||||||
|
genuinely needs a human with dashboard access, not another probe.
|
||||||
|
- **Config prepared** to unblock as soon as that's done: added an
|
||||||
|
`omniroute-search` entry to `~/.qwen/settings.json`'s `mcpServers` (backed up
|
||||||
|
the prior file first as `settings.json.bak-wayfinder-<timestamp>`):
|
||||||
|
```json
|
||||||
|
"omniroute-search": {
|
||||||
|
"httpUrl": "http://proxy-ai.home:20128/api/mcp/stream",
|
||||||
|
"headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Deliberately a separate env var (`OMNIROUTE_SEARCH_KEY`), not reusing
|
||||||
|
`OMNIROUTE_API_KEY`, so the search-scoped key stays distinct from the
|
||||||
|
model-routing key — matches `docs/proxy-key-onboarding.md`'s per-workload
|
||||||
|
key pattern.
|
||||||
|
|
||||||
|
## Resolution (2026-09-05, completed)
|
||||||
|
|
||||||
|
The dashboard steps above turned out to need a different diagnosis than
|
||||||
|
originally guessed — walked through live with a `oma_live_...` management
|
||||||
|
token and a rotating set of `sk-...` provider keys the user supplied:
|
||||||
|
|
||||||
|
- **`/api/providers` (management API) showed zero search providers at all**
|
||||||
|
— not a misconfigured `searxng-search` entry, it simply didn't exist as a
|
||||||
|
connection anymore (9 connections total, all LLM/chat providers). The
|
||||||
|
CHANGELOG at the pinned ref was checked and shows `/v1/search` under active
|
||||||
|
feature development (a `feat(search)` entry adding Firecrawl support), so
|
||||||
|
this wasn't an OmniRoute-side removal of the search system — the row was
|
||||||
|
just gone from this instance's own database (reason unconfirmed: update
|
||||||
|
migration vs. prior manual removal).
|
||||||
|
- **Created it via the API**, not the dashboard UI — `POST /api/providers`
|
||||||
|
turned out to accept the same generic connection schema used for LLM
|
||||||
|
providers: `{"provider":"searxng-search","name":"searxng"}` (Zod-validated;
|
||||||
|
an empty-body POST surfaced the required fields). Then
|
||||||
|
`PATCH /api/providers/<id>` with `{"providerSpecificData":{"baseUrl":"http://search.home/search"}}`
|
||||||
|
set the real URL, replacing the catalog default.
|
||||||
|
- **Verified end-to-end**: `POST /v1/search` with `provider: "searxng-search"`
|
||||||
|
returned real results (5 hits, `search_cost_usd: 0`, `cached: false`,
|
||||||
|
`response_time_ms: 4495`) — confirms `search.home`'s `extra_hosts` mapping
|
||||||
|
in `docker-compose.yml` resolves correctly from inside the OmniRoute
|
||||||
|
container and the whole chain (OmniRoute → SearXNG → results) works.
|
||||||
|
- **`/api/mcp/status` confirmed `scopesEnforced: false`** on this instance —
|
||||||
|
the `mcp:connect`/`execute:search` scope requirement documented upstream
|
||||||
|
isn't actually being enforced here, so any valid provider key connects.
|
||||||
|
`mcpEnabled: true` already, transport `streamable-http`.
|
||||||
|
- **Key rotation caveat hit live**: the first `sk-...` key the user shared
|
||||||
|
went from working to a flat 401 on *every* route (including `/v1/models`)
|
||||||
|
partway through testing — consistent with it having been revoked/rotated
|
||||||
|
server-side. A second key worked immediately. If this setup stops working
|
||||||
|
later, check for exactly this before re-diagnosing the whole chain.
|
||||||
|
- **Final `~/.qwen/settings.json` `mcpServers` entry** (confirmed connected
|
||||||
|
via `qwen mcp list` → `✓ omniroute-search: ... - Connected`):
|
||||||
|
```json
|
||||||
|
"omniroute-search": {
|
||||||
|
"httpUrl": "http://proxy-ai.home/api/mcp/stream",
|
||||||
|
"headers": { "Authorization": "Bearer ${OMNIROUTE_SEARCH_KEY}" }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
`OMNIROUTE_SEARCH_KEY` is exported in `~/.bashrc` alongside the existing
|
||||||
|
`OMNIROUTE_API_KEY`, holding the second (working) `sk-...` key.
|
||||||
|
|
||||||
|
**Status: done.** `qwen` in WSL has a connected `omniroute-search` MCP server
|
||||||
|
backed by this stack's own SearXNG instance — no external search API, no
|
||||||
|
Alibaba DashScope dependency. Not yet exercised: an actual `qwen` chat turn
|
||||||
|
that triggers the `omniroute_web_search` tool call (only the MCP handshake
|
||||||
|
and the raw `/v1/search` call were verified directly).
|
||||||
|
|
||||||
|
## Open questions / unverified
|
||||||
|
|
||||||
|
- **`tools.webSearch.*` settings.json schema** — described on Qwen Code's
|
||||||
|
web-search doc page but absent from the canonical settings-schema page; not
|
||||||
|
independently confirmed (e.g. via `qwen --help` or source) — moot for this
|
||||||
|
setup since the MCP path (above) is what's being wired in, not the
|
||||||
|
DashScope-only built-in tool.
|
||||||
|
- **DashScope vs SearXNG data-residency/cost tradeoffs** — out of scope here,
|
||||||
|
but worth noting the built-in `web_search` tool sends queries to Alibaba
|
||||||
|
Cloud regardless of this stack being otherwise fully self-hosted.
|
||||||
|
- OmniRoute's own docs (already flagged in this repo's `README.md`) describe
|
||||||
|
stealth/anti-detection and TLS-interception features elsewhere in its repo;
|
||||||
|
none of that is exercised by anything in this note, but it's the same caveat
|
||||||
|
`README.md` already carries forward from issue #31.
|
||||||
Reference in New Issue
Block a user