CONTEXT.md

@@ -0,0 +1,59 @@
+# LLM-Server
+
+A self-hosted inference stack: local models served on one GPU machine, reached by other tools through a single authenticated entry point. This file is a glossary only — for how things are configured, follow the links in the docs to the source.
+
+## Language
+
+### Machine
+
+**Server**:
+The one machine that runs the stack.
+_Avoid_: AI box, R9700 box, this machine
+
+**Host**:
+The Docker host, as opposed to a container — only used for that distinction.
+_Avoid_: using it as a synonym for Server
+
+### Serving models
+
+**Backend**:
+A model server process that serves one model to the gateway (`llama-server`, `qwen-classifier`).
+_Avoid_: provider, model server
+
+**Slot**:
+One concurrent request lane inside a backend; a backend handles as many requests at once as it has slots.
+_Avoid_: parallel request, lane
+
+**Main model**:
+The large model that does the actual work for workloads.
+_Avoid_: chat model
+
+**Classifier model**:
+The small model that judges a coding CLI's tool calls before they run.
+_Avoid_: fast model (`fastModel` only when quoting qwen-code's config key)
+
+### Reaching the models
+
+**Gateway**:
+The single entry point every model call goes through (OmniRoute).
+_Avoid_: proxy, AI proxy
+
+**Provider connection**:
+The gateway's registration of one backend.
+_Avoid_: provider (alone), connection (alone)
+
+**API key**:
+A credential issued by the gateway to exactly one workload.
+_Avoid_: virtual key, workload key
+
+**Workload**:
+Anything that calls the gateway with its own API key.
+_Avoid_: client, consumer
+
+**Coding CLI**:
+A workload that is an interactive coding agent (Claude Code, Kimi CLI, OpenCode, Qwen Code).
+_Avoid_: harness
+
+**Reverse proxy**:
+The existing Nginx Proxy Manager instance that publishes the gateway under hostnames; not part of this stack.
+_Avoid_: proxy (alone)

README.md

@@ -1,35 +1,29 @@
 # LLM-Server
 
-Local AI inference stack: llama.cpp (ROCm) serving Qwen3.8-27B on an AMD Radeon AI PRO R9700, fronted by the OmniRoute AI gateway, with Lazytainer auto-suspending the inference container when idle.
+> **Source of truth:** [`docker-compose.yml`](docker-compose.yml) (what runs) and [`scripts/update.sh`](scripts/update.sh) (how it's deployed). The docs explain *why*; if they disagree with the source, the source wins.
 
-See the wayfinder map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/issues/1)) for the full architecture rationale and open questions.
+A self-hosted inference stack on one AMD GPU server. Two llama.cpp **backends**, the main model and the classifier model, sit behind an OmniRoute **gateway**. Every **workload**, mostly coding CLIs, reaches them through the gateway with its own **API key**. Terms are defined in [`CONTEXT.md`](CONTEXT.md).
 
 ## Quickstart
 
+On the server:
+
 ```bash
 ./scripts/update.sh
 ```
 
-`update.sh` creates `.env` from `.env.example` if missing, fills in every random secret it can generate itself (via `openssl`, `SEARXNG_LAN_IP` resolved from `search.home` on this host), downloads the model GGUF into the `models` volume if it's not there yet, then pulls/builds/brings up the whole stack. Safe to re-run any time — it only fills in what's still blank, skips the model if already downloaded, and only recreates what changed.
+It creates and syncs `.env`, generates secrets, downloads the models, and brings the stack up. It's safe to re-run at any time. What it does and why: [`docs/operations.md`](docs/operations.md).
 
-llama.cpp's own API is internal-only — everything routes through the AI gateway below.
+## Docs
 
-Pointing Claude Code CLI, Kimi CLI, OpenCode CLI, or Qwen Code CLI at the local endpoint: see [`docs/coding-cli-setup/`](docs/coding-cli-setup/index.md).
+| Doc | What it covers |
-
+|---|---|
-**Known risk**: Qwen3.8-27B's tool-calling reliability against llama.cpp's Anthropic shim is not yet verified (open upstream parser bugs against its model lineage) — see `docs/research/qwen3.8-27b-tool-calling.md`.
+| [`CONTEXT.md`](CONTEXT.md) | Glossary: the terms every doc uses |
-
+| [`docs/architecture.md`](docs/architecture.md) | Each service, why it exists, and why it's configured the way it is |
-## AI gateway (OmniRoute)
+| [`docs/gateway.md`](docs/gateway.md) | OmniRoute: dashboard, provider connections, API keys, timeouts, web search |
-
+| [`docs/network-access.md`](docs/network-access.md) | Who can reach which port, and why |
-An [AI gateway/proxy](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) fronts llama.cpp: per-workload API keys and usage tracking. As of [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) this is [OmniRoute](https://github.com/diegosouzapw/OmniRoute), replacing the original LiteLLM setup. `./scripts/update.sh` handles most of OmniRoute's secrets (see `.env.example`); per-workload API keys still need minting by hand in the dashboard — see [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md).
+| [`docs/operations.md`](docs/operations.md) | What `update.sh` does: env sync, secrets, downloads |
-
+| [`docs/coding-cli-setup/`](docs/coding-cli-setup/index.md) | Pointing Claude Code, Kimi CLI, OpenCode and Qwen Code at the gateway |
-- Gateway API: `http://<this-machine>:${OMNIROUTE_PORT:-4000}/v1` locally, or `proxy-ai.home` / `proxy-ai.haylan.ch` once routed through NPM — see [`docs/network-access.md`](docs/network-access.md).
+| [`docs/research/`](docs/research/README.md) | Dated research and decision records, each marked current or superseded |
-- Dashboard (key/provider management): LAN/host-only, never published to the internet — see `docs/network-access.md`.
+| [`docs/server-planing.md`](docs/server-planing.md) | Background research: a 4× GPU rack build. Not an active plan |
-- Issuing a key for a new workload: [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md).
+| [`docs/agents/`](docs/agents/) | Instructions for coding agents: [issue tracker](docs/agents/issue-tracker.md), [domain docs](docs/agents/domain.md), [writing docs](docs/agents/writing-docs.md) |
-
-Coding CLIs (see [`docs/coding-cli-setup/`](docs/coding-cli-setup/index.md)) route through the gateway — llama-server has no published host port. **Not yet verified**: none of this has been smoke-tested on real hardware yet — see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31)'s tickets for the open items (provider registration, per-workload key minting).
-
-**Note on this choice**: OmniRoute's own docs (`docs/security/STEALTH_GUIDE.md`, `MITM-TPROXY-DECRYPT.md`, `PUBLIC_CREDS.md` in its repo) describe shipped features for evading AI-provider client detection, system-wide HTTPS interception via a locally-installed root CA, and hiding credentials from secret scanners. None of that is used by this stack's configuration, but it's a real characteristic of the upstream project — see issue #31's Notes for the full research trail before extending this integration further.
-
-### Web search
-
-The gateway also fronts SearXNG-backed web search — see `docs/research/litellm-searxng-search.md` for the original research (still applicable — same standalone-endpoint pattern, see issue #31's #35).

docs/architecture.md

@@ -0,0 +1,54 @@
+# Architecture
+
+> **Source of truth:** [`docker-compose.yml`](../docker-compose.yml) → each service below. This page explains *why*; flags, values and defaults live in the source. If they disagree, the source wins.
+
+Terms as defined in [`CONTEXT.md`](../CONTEXT.md).
+
+Every workload calls the **gateway** (`omniroute`) with its own API key. The gateway forwards each call over a **provider connection** to one of two **backends** on the one GPU: `llama-server` (main model) or `qwen-classifier` (classifier model). `qdrant` sits beside them on the same Docker network. It isn't a model, so it isn't behind the gateway.
+
+## GPU access
+
+Both backends share the one GPU, and both carry the same GPU settings in compose:
+
+- **`group_add` uses numeric GIDs, not group names.** Docker resolves a *named* `group_add` entry against the container's own `/etc/group`, not the host's. That fails unpredictably when the image doesn't define the group, and it gets worse when several GPU services start at once. `scripts/update.sh` resolves the host's real GIDs into `HOST_VIDEO_GID` / `HOST_RENDER_GID` (see [operations → Host-resolved values](operations.md#host-resolved-values)).
+- **`GPU_MAX_HW_QUEUES`** works around [ROCm/ROCm#5706](https://github.com/ROCm/ROCm/issues/5706): the card sits pinned at full load and boost clock whenever two HIP contexts touch it. With two backends running, there are always two.
+
+Background: [`research/rocm-gpu-pin-and-render-group.md`](research/rocm-gpu-pin-and-render-group.md). Its service names are older than the current ones, but the findings still apply.
+
+## llama-server
+
+The main model's backend. It is always running. Lazytainer used to idle-stop it, but was removed because its idle detection could never fire behind the gateway ([`research/lazytainer-omniroute-idle-stop.md`](research/lazytainer-omniroute-idle-stop.md)).
+
+- **Model file** (`LLAMA_MODEL_FILE`): the quant choice and its VRAM budget are in [`research/qwen3.8-27b-quant.md`](research/qwen3.8-27b-quant.md).
+- **`LLAMA_GPU_LAYERS`**: the default puts every layer on the GPU. The model is dense, not Mixture-of-Experts, so this is the only RAM-offload knob. `--n-cpu-moe`, `--cpu-moe` and `--override-tensor "exps"` do nothing here. There is no SSD tier to enable either. llama.cpp mmaps the model file, so if GPU and RAM ever can't hold it, the OS pages the rest in from disk as a slow, implicit last resort. Explicit tiered offload is an [open llama.cpp request](https://github.com/ggml-org/llama.cpp/discussions/12507).
+- **`LLAMA_CTX_SIZE`**: set to the model's own maximum (`max_position_embeddings`). It only fits beside the weights because the KV cache is quantized (`--cache-type-k/v`). An fp16 KV cache at that size would not fit the card. The numbers are in the quant research doc.
+- **`LLAMA_PARALLEL`** is the backend's number of **slots**. It's kept low on purpose: more slots meant more contended prefill, which pushed requests past the gateway's timeouts ([gateway → Timeouts](gateway.md#timeouts)). Requests beyond the slot count wait in llama.cpp's own queue, which has no timeout. So the timeout that matters is the gateway's per-connection one.
+- **Per-slot context = `LLAMA_CTX_SIZE / LLAMA_PARALLEL`.** llama.cpp divides the context across slots, and each request gets one slot's share. Every coding CLI's context-window setting must use this number, not the raw context size. Real sessions have come close to it, so check that the per-slot figure stays above observed usage before lowering either value.
+- **`--cache-reuse`** reuses cached KV for any matching prompt chunk instead of re-prefilling from scratch. That fixes the root cause behind the gateway's non-ping SSE timeout ([`research/omniroute-non-ping-sse-stream-timeout.md`](research/omniroute-non-ping-sse-stream-timeout.md)). It pairs with OmniRoute's prompt-cache affinity (a dashboard setting, on by default), which keeps a conversation on the same slot so that there is a matching prefix to reuse.
+- **`--jinja`** enables the chat template that tool calling depends on. Without it, tool-use requests fail outright.
+- **Tool-calling reliability** of this model family on llama.cpp has open upstream parser bugs: [`research/qwen3.8-27b-tool-calling.md`](research/qwen3.8-27b-tool-calling.md).
+- **No host port**: see [network access → Backends](network-access.md#backends).
+
+## qwen-classifier
+
+The classifier model's backend. It serves qwen-code's Auto Mode classifier (`fastModel`), which judges every tool call before it runs.
+
+- **Why a separate backend:** when the classifier was an alias of the main model, every classification queued behind whatever heavy generation already held the main backend's slots, sometimes for minutes.
+- **Why GPU, not CPU:** a CPU-only instance was tried first. Real classification calls carry a conversation transcript, and they ran past the gateway's timeout and retry-looped. It now runs on the GPU with a partial layer count (`--n-gpu-layers`), which leaves VRAM headroom for the main model.
+- **Context size:** qwen-code hard-caps the classifier transcript in its own source (`MAX_TRANSCRIPT_MESSAGES`, `MAX_HISTORICAL_ACTION_CHARS` in `packages/core/src/permissions/classifier-transcript.ts`). `--ctx-size` gives margin over that worst case, and nothing more. The backend has a single slot, so its whole context is per-slot.
+- **VRAM:** measured usage came out well above the weights+KV estimate. The cause was a missing `--flash-attn`: without it, the unfused attention buffers grow roughly quadratically with context. Batch size was the wrong lever. Check with `rocm-smi` after any change to the context size or the model.
+- **`--reasoning off`** is a no-cost safety net against [llama.cpp#20809](https://github.com/ggml-org/llama.cpp/issues/20809). Some builds misdetect Qwen3-Instruct-2507 as a thinking model and route tool calls into `reasoning_content`. Nobody has confirmed whether the current image still needs it.
+- **Model choice:** the smallest Qwen3 with long native context and no RoPE scaling. See [`research/fast-model-choice.md`](research/fast-model-choice.md) and [`research/classifier-context-window-scaling.md`](research/classifier-context-window-scaling.md).
+- Its provider connection needs specific gateway settings: see [gateway → Provider connections](gateway.md#provider-connections).
+
+## omniroute
+
+The gateway. Its runtime configuration (provider connections, API keys) lives in its own database and is managed through the dashboard, not in this repo: see [`gateway.md`](gateway.md).
+
+- **`OMNIROUTE_MEMORY_MB` + `mem_limit`**: the default heap is sized for dashboard-only use. OmniRoute's Docker guide recommends more for coding-agent traffic, and `mem_limit` is the matching container ceiling.
+- **`stop_grace_period`**: SQLite needs time on shutdown to checkpoint its WAL back into the main database file. If this is shortened, recent writes (keys, provider connections) can be lost.
+- **Healthcheck is a TCP connect**, not an HTTP probe. The image has only `node` (no curl, wget or python), and OmniRoute's Docker guide accepts a TCP probe as a liveness check.
+
+## qdrant
+
+A vector store. It has no authentication, so its port is bound to the server's loopback interface only: see [network access → qdrant](network-access.md#qdrant).

docs/coding-cli-setup/claude-code.md

@@ -2,12 +2,14 @@
 
 [← back to overview](index.md)
 
+> **Source of truth:** [Claude Code's docs](https://docs.claude.com/en/docs/claude-code/overview) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
+
 Claude Code speaks the **Anthropic Messages API** — point it at the gateway's unified endpoint, not llama.cpp directly:
 
 ```bash
-export ANTHROPIC_BASE_URL=http://<ai-box>:${OMNIROUTE_PORT:-4000}
+export ANTHROPIC_BASE_URL=http://<server>:<OMNIROUTE_API_PORT>
-export ANTHROPIC_API_KEY=<claude-code-cli virtual key>
+export ANTHROPIC_API_KEY=<claude-code-cli API key>
 claude
 ```
 
-Requires llama.cpp's `--jinja` flag (already set in `docker-compose.yml`) — without it, tool-use requests fail outright.
+Relies on the backend's `--jinja` flag ([architecture → llama-server](../architecture.md#llama-server)).

docs/coding-cli-setup/index.md

@@ -1,34 +1,29 @@
-# Pointing a coding-agent CLI at this stack
+# Pointing a coding CLI at this stack
 
 [← back to README](../../README.md)
 
-This stack routes through the [AI gateway](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) (OmniRoute, see issue #31) rather than talking to llama.cpp directly — llama.cpp's own port is internal-only now (see `docker-compose.yml`). The gateway exposes:
+> **Source of truth:** each CLI's own config docs (linked per page), plus [`docker-compose.yml`](../../docker-compose.yml) → `omniroute` for the gateway port. This page explains *how to wire them up*; ports and model settings live in those sources. If they disagree, the sources win.
 
-- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1`
+Terms as defined in [`CONTEXT.md`](../../CONTEXT.md). Every coding CLI is a workload: it reaches the models through the gateway, never a backend directly ([network access](../network-access.md)). The gateway speaks two wire formats for the same models:
-- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_PORT:-4000}`
 
-Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, registered in the gateway (naming is yours to pick when adding the llama-cpp provider connection — these docs assume `qwen3.8-27b-local` for continuity) — behind whichever wire format the client speaks.
+- **OpenAI-compatible:** `http://<server>:<OMNIROUTE_API_PORT>/v1`
+- **Anthropic Messages:** `http://<server>:<OMNIROUTE_API_PORT>` (OmniRoute's `/v1/messages`)
 
-`<ai-box>` is this machine's LAN address, or `proxy-ai.home` if your local DNS resolves that hostname directly to the box — see `docs/network-access.md`. If you're running a coding CLI from this machine itself, `localhost` works too.
+`<server>` is the server's LAN address or `proxy-ai.home`, or `localhost` on the server itself. The main model's name is whatever it was registered as in the gateway. These docs use `qwen3.8-27b-local` ([gateway → Provider connections](../gateway.md#provider-connections)).
 
-**Each CLI needs its own virtual key** — create one per docs/proxy-key-onboarding.md (omniroute's dashboard, `<workload>-<purpose>` naming, e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli`). No budget set by default. These are the machine's interactive/high-priority workloads per `docs/proxy-request-priority.md`.
+**Each CLI needs its own API key**, labelled e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli` ([gateway → API keys](../gateway.md#api-keys)).
 
-> **Read this before relying on it for real work.** Qwen3.8-27B's tool-calling has **documented, open llama.cpp upstream bugs** (parser fails on text before `<tool_call>`, tool calls emitted as inert XML inside thinking blocks — see `docs/research/qwen3.8-27b-tool-calling.md`). Every CLI below inherits this risk identically, regardless of wire format. Don't trust it for unattended multi-step agentic work until you've run the smoke test in [issue #5](https://git.arthurerlich.de/haylan/LLM-Server/issues/5) (and the proxy-specific smoke test in [issue #17](https://git.arthurerlich.de/haylan/LLM-Server/issues/17)).
+**Context window:** wherever a CLI asks for the main model's context size, use the per-slot context, `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from the server's `.env`, not the raw context size ([architecture → llama-server](../architecture.md#llama-server)).
+
+> **Tool-calling risk.** The main model's tool calling has open llama.cpp upstream parser bugs ([`research/qwen3.8-27b-tool-calling.md`](../research/qwen3.8-27b-tool-calling.md)). Every CLI below inherits that risk, whatever its wire format. Test it on your own workflow before relying on it for unattended multi-step agent work.
 
 ## Per-CLI setup
 
-- [Claude Code CLI](claude-code.md)
+| CLI | Wire format | Config |
-- [Kimi CLI](kimi-cli.md)
+|---|---|---|
-- [OpenCode CLI](opencode.md)
+| [Claude Code](claude-code.md) | Anthropic Messages | `ANTHROPIC_BASE_URL` env var |
-- [Qwen Code CLI](qwen-code.md)
+| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `config.toml` provider block |
+| [OpenCode](opencode.md) | OpenAI Chat Completions | `opencode.json` provider block |
+| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (main model + classifier model) | `~/.qwen/settings.json` `modelProviders.openai` |
 
-## Summary
+If requests fail with a semaphore timeout under concurrent load, see [gateway → Timeouts](../gateway.md#timeouts).
-
-| CLI | Wire format | Endpoint | Config |
-|---|---|---|---|
-| [Claude Code](claude-code.md) | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_PORT:-4000}` | `ANTHROPIC_BASE_URL` env var |
-| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `config.toml` provider block |
-| [OpenCode](opencode.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `opencode.json` provider block |
-| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (2 models: chat + `fastModel`) | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `~/.qwen/settings.json` `modelProviders.openai` |
-
-Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/proxy-key-onboarding.md`, `docs/research/omniroute-account-semaphore-timeout.md` (a connection that can only handle a few concurrent requests — like `llama-server` or `qwen-classifier` — hits a hardcoded 30s reject once more requests queue up than its `maxConcurrent`, unless configured around it).

docs/coding-cli-setup/kimi-cli.md

@@ -2,13 +2,15 @@
 
 [← back to overview](index.md)
 
+> **Source of truth:** [Kimi CLI's repo](https://github.com/MoonshotAI/kimi-cli) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
+
 Kimi CLI speaks plain **OpenAI Chat Completions**. Configure a provider block in its config file (`config.toml`):
 
 ```toml
 [providers.openai]
 type = "openai"
-base_url = "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
+base_url = "http://<server>:<OMNIROUTE_API_PORT>/v1"
-api_key = "<kimi-cli virtual key>"
+api_key = "<kimi-cli API key>"
 ```
 
 If Kimi CLI's response parsing gets confused by Qwen's `<think>...</think>` reasoning tags, check its `reasoning_key` setting — it's configurable for non-standard local server responses.

docs/coding-cli-setup/opencode.md

@@ -2,6 +2,8 @@
 
 [← back to overview](index.md)
 
+> **Source of truth:** [OpenCode's docs](https://opencode.ai/docs/) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
+
 Confirmed project: **`anomalyco/opencode`** (renamed from `sst/opencode` — don't confuse with the unrelated `opencode-ai/opencode` Go TUI). Docs: https://opencode.ai/docs/
 
 **Install**:
@@ -15,17 +17,17 @@ curl -fsSL https://opencode.ai/install | bash
 {
   "$schema": "https://opencode.ai/config.json",
   "provider": {
-    "aiproxy": {
+    "llm-server": {
       "npm": "@ai-sdk/openai-compatible",
-      "name": "AI proxy (local)",
+      "name": "LLM-Server gateway",
       "options": {
-        "baseURL": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
+        "baseURL": "http://<server>:<OMNIROUTE_API_PORT>/v1",
-        "apiKey": "<opencode-cli virtual key>"
+        "apiKey": "<opencode-cli API key>"
       },
       "models": {
         "qwen3.8-27b-local": {
           "name": "Qwen3.8-27B",
-          "limit": { "context": 131072, "output": 8192 }
+          "limit": { "context": <LLAMA_CTX_SIZE / LLAMA_PARALLEL>, "output": 8192 }
         }
       }
     }
@@ -33,13 +35,13 @@ curl -fsSL https://opencode.ai/install | bash
 }
 ```
 
-Set `limit.context` to the *per-slot* context this stack actually serves — `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from `.env` (262144 / 2 = 131072 by default), not raw `LLAMA_CTX_SIZE` and not a value assumed from the model card: llama.cpp divides `--ctx-size` across concurrent slots, so each request only gets one slot's share. OpenCode uses this for its own context-management bookkeeping, not the server.
+`limit.context` is the per-slot context ([overview](index.md)). OpenCode uses it for its own context bookkeeping; the server doesn't read it.
 
-Select the model with `aiproxy/qwen3.8-27b-local`.
+Select the model with `llm-server/qwen3.8-27b-local`.
 
 **OpenCode-specific risks** (on top of the shared Qwen3.8-27B tool-calling risk — see [overview](index.md)):
-- Requires llama.cpp's `--jinja` flag (already set) — without it, OpenCode's unconditional tool-call scaffolding gets a 500.
+- Relies on the backend's `--jinja` flag ([architecture → llama-server](../architecture.md#llama-server)); without it, OpenCode's unconditional tool-call scaffolding gets a 500.
 - [anomalyco/opencode#20669](https://github.com/anomalyco/opencode/issues/20669) (closed as "not planned" — a live, unfixed risk): OpenCode's `bash` tool crashes if the model omits the optional `description` field on a tool call; some local backends return `finish_reason: tool_calls` with an empty array, which can hang the agent loop instead of stopping cleanly.
 - Thinking-mode handling (`options.reasoningEffort`) is undocumented for models that emit inline `<think>` tags rather than a native reasoning API field — expect no effect from that config on this model; untested.
 
-Further reading: `docs/research/opencode-cli-setup.md`.
+Further reading: [`research/opencode-cli-setup.md`](../research/opencode-cli-setup.md) (its base URL example predates the gateway), [`research/opencode-auto-compact.md`](../research/opencode-auto-compact.md).

docs/coding-cli-setup/qwen-code.md

@@ -2,16 +2,13 @@
 
 [← back to overview](index.md)
 
-Qwen Code speaks plain **OpenAI Chat Completions**, and — unlike the other CLIs — needs *two* models: the main chat model, and a `fastModel` for Auto Mode's action classifier. Both are registered as separate providers in OmniRoute but reachable through the same gateway URL.
+> **Source of truth:** [Qwen Code's repo](https://github.com/QwenLM/qwen-code) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
+
+Qwen Code speaks plain **OpenAI Chat Completions**. Unlike the other CLIs, it needs *two* models: the main model, and the classifier model as its `fastModel` for Auto Mode's action classifier. Each has its own provider connection in the gateway, and both are reached through the same gateway URL.
 
 ## Why a second model exists
 
-Auto Mode's action classifier (`permissions.autoMode`) is qwen-code's per-tool-call safety gate — it decides whether to auto-approve or block a shell command / tool call before it runs. It was originally aliased onto the main 27B model's own OmniRoute connection. That broke two ways in practice (see `docs/research/fast-model-choice.md` for the model research, and the issue-tracker history for the full incident):
+Auto Mode's action classifier (`permissions.autoMode`) is qwen-code's safety gate for every tool call: it decides whether to auto-approve or block a shell command or tool call before it runs. It runs on its own backend, `qwen-classifier`. Why that backend exists and how it's sized: [architecture → qwen-classifier](../architecture.md#qwen-classifier).
-
-- **Queued behind heavy work.** Every classification call competed for the main model's 2 GPU slots with whatever real generation was already running, so a classifier check could sit blocked for minutes.
-- **CPU-only was tried first and was too slow.** Isolating the classifier onto its own CPU-only llama.cpp instance avoided the GPU queue entirely, but real classification calls (which can carry a non-trivial conversation transcript, not just the bare tool call) blew past OmniRoute's request timeout and retry-looped.
-
-The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.yml`) running a small model (`Qwen3-4B-Instruct-2507`) on its own **partial** GPU offload — enough layers on the R9700 to be fast, sized to leave real VRAM headroom next to the 27B model rather than trusting a naive weights+KV estimate (see that service's comment block in `docker-compose.yml` for the actual measured numbers and the two wrong turns — batch-size tuning, then flash-attn — before partial offload turned out to be the real lever).
 
 ## `~/.qwen/settings.json`
 
@@ -23,16 +20,16 @@ The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.ym
         "id": "<main-model-provider-id-in-omniroute>",
         "name": "qwen3.8-27b-local",
         "envKey": "OMNIROUTE_API_KEY",
-        "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
+        "baseUrl": "http://<server>:<OMNIROUTE_API_PORT>/v1",
-        "generationConfig": { "contextWindowSize": 131072 }
+        "generationConfig": { "contextWindowSize": <LLAMA_CTX_SIZE / LLAMA_PARALLEL> }
       },
       {
         "id": "<classifier-provider-id-in-omniroute>",
         "name": "qwen3-4b-classifier",
         "envKey": "OMNIROUTE_API_KEY",
-        "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
+        "baseUrl": "http://<server>:<OMNIROUTE_API_PORT>/v1",
         "generationConfig": {
-          "contextWindowSize": 65536,
+          "contextWindowSize": <qwen-classifier --ctx-size>,
           "extra_body": { "chat_template_kwargs": { "enable_thinking": false } }
         }
       }
@@ -41,21 +38,21 @@ The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.ym
   "security": { "auth": { "selectedType": "openai" } },
   "model": {
     "name": "<main-model-provider-id-in-omniroute>",
-    "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
+    "baseUrl": "http://<server>:<OMNIROUTE_API_PORT>/v1"
   },
   "fastModel": "<classifier-provider-id-in-omniroute>"
 }
 ```
 
-- `envKey` names the environment variable Qwen Code reads the virtual key from — set `OMNIROUTE_API_KEY=<qwen-code-cli virtual key>` before launching. Both providers can share one virtual key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls.
+- `envKey` names the environment variable Qwen Code reads the API key from — set `OMNIROUTE_API_KEY=<qwen-code-cli API key>` before launching. Both providers can share one API key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls.
-- **`contextWindowSize` for the main model is per-slot, not `LLAMA_CTX_SIZE` itself** — llama.cpp divides `--ctx-size` across `LLAMA_PARALLEL` concurrent slots, and each request only gets one slot's share (same correction applies to OpenCode's `limit.context`). Compute it from `.env`: `LLAMA_CTX_SIZE / LLAMA_PARALLEL` = `262144 / 2` = **131072**.
+- **The main model's `contextWindowSize` is the per-slot context**, not `LLAMA_CTX_SIZE` itself ([overview](index.md)).
-- **The classifier's `contextWindowSize` (65536) is not per-slot math** — `qwen-classifier` runs `--parallel 1`, so its whole `--ctx-size` belongs to the one slot. 65536 isn't a guess either: qwen-code's own source hard-caps the classifier transcript (`MAX_TRANSCRIPT_MESSAGES=40`, `MAX_HISTORICAL_ACTION_CHARS=4000`/message in `packages/core/src/permissions/classifier-transcript.ts`) — worst case is ~40-50K tokens, so 65536 gives real margin without wasting VRAM the way the original 131072 (copied from the main model's entry, not an actual qwen-code requirement) would have.
+- **The classifier's `contextWindowSize` is its whole `--ctx-size`.** It has a single slot, so there's no per-slot division ([architecture → qwen-classifier](../architecture.md#qwen-classifier)).
-- `enable_thinking: false` on the classifier matters for parseability, though `Qwen3-4B-Instruct-2507` is already architecturally non-thinking (see `fast-model-choice.md` §3) — this is belt-and-suspenders for any future fast-model swap that isn't.
+- `enable_thinking: false` on the classifier matters for parseability, though `Qwen3-4B-Instruct-2507` is already architecturally non-thinking (see [`research/fast-model-choice.md`](../research/fast-model-choice.md) §3) — this is belt-and-suspenders for any future fast-model swap that isn't.
 - Qwen Code also recognizes `advisorModel`, `visionModel`, `compactionModel`, `imageModel` for other model roles — none are wired up in this stack; only `fastModel` is required.
 
 ## Web search via OmniRoute
 
-Qwen Code's own built-in web search (`tools.webSearch.enabled`) has nothing to search with here — leave it `false`. Instead this stack's SearXNG-backed search (README §"Web search") is exposed through a thin stdio MCP wrapper around OmniRoute's `/v1/search` REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. `~/.qwen/mcp-servers/omniroute-search/index.mjs` (needs `@modelcontextprotocol/sdk` and `zod`: `npm init -y && npm i @modelcontextprotocol/sdk zod` in that directory):
+Qwen Code's own built-in web search (`tools.webSearch.enabled`) has nothing to search with here — leave it `false`. Instead this stack's SearXNG-backed search ([gateway → Web search](../gateway.md#web-search)) is exposed through a thin stdio MCP wrapper around OmniRoute's `/v1/search` REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. `~/.qwen/mcp-servers/omniroute-search/index.mjs` (needs `@modelcontextprotocol/sdk` and `zod`: `npm init -y && npm i @modelcontextprotocol/sdk zod` in that directory):
 
 ```js
 import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
@@ -104,13 +101,13 @@ Register it in `~/.qwen/settings.json`:
 }
 ```
 
-It reuses the same `OMNIROUTE_API_KEY` env var as the model providers above — the virtual key needs search permission in OmniRoute, not just chat-completions.
+It reuses the same `OMNIROUTE_API_KEY` env var as the model providers above — the API key needs search permission in OmniRoute, not just chat-completions.
 
 **Non-interactive mode (`qwen -p ...`) needs this tool explicitly allow-listed.** MCP tools require interactive confirmation by default; `--approval-mode auto` alone doesn't bypass that for a non-interactive run — pass `--allowed-tools mcp__omniroute-search__search` (or `-y` for full YOLO) alongside `-p`, or the search call never reaches the classifier at all and silently no-ops. Confirmed live: without the allow-list, only the tool calls the CLI's non-interactive gate lets through end up as classifier requests.
 
 ## Auto Mode tuning
 
-Auto Mode's action classifier calls the fast model above. Even on the dedicated GPU-resident instance, give it real timeout headroom rather than trusting OmniRoute's default — and since this stack is a single trusted local proxy, it's reasonable to pre-approve requests to it rather than confirm every call:
+Auto Mode's action classifier calls the classifier model above. Even on its own GPU-resident backend, give it real timeout headroom rather than trusting OmniRoute's default. And since the gateway is a single trusted local endpoint, it's reasonable to pre-approve requests to it rather than confirm every call:
 
 ```json
 {
@@ -125,6 +122,6 @@ Auto Mode's action classifier calls the fast model above. Even on the dedicated
 
 `hints.allow` entries are free-text descriptions the classifier matches against, not exact strings — capped at 150 entries/200 chars each.
 
-Also set a generous per-connection timeout on the classifier's own OmniRoute provider connection (`providerSpecificData.timeoutMs`, dashboard or `PATCH /api/providers/{id}` — not a `.env` value, see `docs/network-access.md` for reaching the dashboard API). 120000ms is comfortable for the current GPU-resident setup (real measured latency: well under a second for a short check, low seconds for the largest realistic transcript) — this isn't the 20-minute figure the main 27B connection needs, since the classifier isn't competing for a contended GPU slot the way the main model can.
+Also set a generous per-connection timeout on the classifier's own OmniRoute provider connection (`providerSpecificData.timeoutMs`, dashboard or `PATCH /api/providers/{id}` — not a `.env` value, see [gateway → Dashboard](../gateway.md#dashboard)). 120000ms is comfortable for the current GPU-resident setup (real measured latency: well under a second for a short check, low seconds for the largest realistic transcript) — this isn't the 20-minute figure the main 27B connection needs, since the classifier isn't competing for a contended GPU slot the way the main model can.
 
 Everything else in `~/.qwen/settings.json` (`hooks`, `security.auth`'s underlying tooling, editor prefs) is per-machine, not part of pointing at this stack — don't copy it wholesale between machines.

docs/gateway.md

@@ -0,0 +1,62 @@
+# Gateway (OmniRoute)
+
+> **Source of truth:** [`docker-compose.yml`](../docker-compose.yml) → `omniroute` service, and [`.env.example`](../.env.example) → OmniRoute section. Provider connections and API keys live in OmniRoute's own database, managed through its dashboard, not in this repo. This page explains *why* and *how*; values live in those sources. If they disagree, the source wins.
+
+Terms as defined in [`CONTEXT.md`](../CONTEXT.md). How the gateway fits the rest of the stack: [`architecture.md`](architecture.md#omniroute).
+
+## Dashboard
+
+Browse to `http://<server>:<OMNIROUTE_DASHBOARD_PORT>` from the LAN. The dashboard isn't routed to the internet: see [network access → Gateway](network-access.md#gateway). For the first login, use `OMNIROUTE_INITIAL_PASSWORD` from the server's `.env`, then change the password in the dashboard.
+
+## Provider connections
+
+Each backend is registered once in the dashboard (Providers → add a **llama.cpp** provider) or via `POST /api/providers`. The base URL is `http://<service>:<container port>` on the stack network, using the service name and port from compose. None of this is checked into the repo. These docs assume the main model is registered as `qwen3.8-27b-local`.
+
+- **`OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` / `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS`** have to be on, because backend URLs are private addresses. They relax OmniRoute's SSRF guard, which normally blocks registering a provider at a private or local address. Anyone with dashboard access can therefore point the gateway at any host on the stack network or the LAN.
+- **Classifier connection:** turn OmniRoute's memory-tool injection off for it. Injected tools break the classifier's output ([`research/omniroute-memory-tool-injection-breaks-classifier.md`](research/omniroute-memory-tool-injection-breaks-classifier.md)). Also give it a generous per-connection timeout (see [Timeouts](#timeouts)).
+- Background on registering local backends: [`research/omniroute-qwen-websearch.md`](research/omniroute-qwen-websearch.md).
+
+## API keys
+
+Every workload gets its own API key, so each has its own visible usage.
+
+**Create**
+
+1. [Open the dashboard](#dashboard) → "Keys" → "Create API key".
+2. Label it `<workload>-<purpose>`, e.g. `paperless-ocr`, `claude-code-cli`. The dashboard lists keys by label, so the label list is the ledger. There's no separate tracking doc.
+3. Copy the key value. OmniRoute shows it only once.
+
+Keys can only be minted by hand for now. `POST /api/keys` needs a dashboard login session (`ManagementSessionAuth`), not a static bearer key, and that flow hasn't been verified against a live instance.
+
+**Hand over:** put the key in that workload's own config or `.env`, never in this repo. A key used for [web search](#web-search) needs search permission, not just chat completions.
+
+**Revoke** ("Keys" → delete) when the workload is retired or the key may have leaked, then remove it from the workload's config. There's no scheduled rotation.
+
+## Timeouts
+
+A slow request can be cut off at several layers. Each setting is named below, and its value lives where the list says:
+
+- **`STREAM_IDLE_TIMEOUT_MS`** (`.env.example`): the longest allowed gap *between* SSE chunks once streaming has started.
+- **`REQUEST_TIMEOUT_MS`** (`.env.example`): the wait budget for the *first* non-ping SSE event. Retries and cooldowns eat into it, so the effective deadline is whatever budget remains, not a flat timer. See [`research/omniroute-non-ping-sse-stream-timeout.md`](research/omniroute-non-ping-sse-stream-timeout.md).
+- **Per-connection `providerSpecificData.timeoutMs`** (dashboard only, or `PATCH /api/providers/{id}`): bounds a request that is waiting behind the backend's busy slots.
+- **Per-connection concurrency semaphore**: a hardcoded reject once more requests queue than the connection's `maxConcurrent`. It is not configurable, only avoidable: [`research/omniroute-account-semaphore-timeout.md`](research/omniroute-account-semaphore-timeout.md).
+- **`OMNIROUTE_DIRECT_HEADERS_TIMEOUT_MS`** (not set in this repo): [`research/omniroute-direct-response-timeout-outage-2026-09-15.md`](research/omniroute-direct-response-timeout-outage-2026-09-15.md).
+
+These are raised above OmniRoute's defaults because contended prefill on the main backend can take a long time before the first token ([architecture → llama-server](architecture.md#llama-server)).
+
+## Web search
+
+SearXNG runs elsewhere on the LAN as `search.home`. It's registered as a search provider connection and exposed through the gateway's `/v1/search` endpoint. The container can't resolve `search.home` (it's a LAN DNS name), so compose adds an `extra_hosts` entry from `SEARXNG_LAN_IP`, which `update.sh` resolves on the server ([operations → Host-resolved values](operations.md#host-resolved-values)).
+
+- Wiring it into Qwen Code: [`coding-cli-setup/qwen-code.md`](coding-cli-setup/qwen-code.md#web-search-via-omniroute).
+- Research: [`research/omniroute-qwen-websearch.md`](research/omniroute-qwen-websearch.md) (current setup), [`research/dashscope-websearch-selfhost-options.md`](research/dashscope-websearch-selfhost-options.md) (why DashScope isn't emulated).
+
+## Upstream caution
+
+OmniRoute's own docs (`docs/security/STEALTH_GUIDE.md`, `MITM-TPROXY-DECRYPT.md` and `PUBLIC_CREDS.md` in [its repo](https://github.com/diegosouzapw/OmniRoute)) describe shipped features for:
+
+- evading AI-provider client detection
+- intercepting HTTPS system-wide via a locally installed root CA
+- hiding credentials from secret scanners
+
+This stack's configuration uses none of them. They are a real characteristic of the upstream project, though, so weigh them before extending this integration.

docs/network-access.md

@@ -1,24 +1,29 @@
-# Network access: proxy-ai.home / proxy-ai.haylan.ch
+# Network access
 
-This stack has no chat UI — every client is a coding CLI reaching the AI gateway (OmniRoute). It doesn't run its own reverse proxy — it publishes the gateway's API port to the host and relies on the **existing Nginx Proxy Manager (NPM)** instance already fronting other self-hosted services on this network.
+> **Source of truth:** [`docker-compose.yml`](../docker-compose.yml) → the `ports:` / `expose:` entries of each service. This page explains *who can reach what, and why*; port numbers live in the source. If they disagree, the source wins.
 
-## llama.cpp's raw API stays LAN-only — deliberately
+Terms as defined in [`CONTEXT.md`](../CONTEXT.md). The stack doesn't run its own reverse proxy. The existing **reverse proxy** (Nginx Proxy Manager, not part of this stack) publishes the gateway under hostnames.
 
-The inference API (port `${LLAMA_PORT:-8080}`) is **not** registered in NPM and is **not** reachable externally. It has no authentication of its own — putting it on the public internet would mean an unauthenticated inference endpoint. Coding-agent CLIs (Claude Code, Kimi, OpenCode, Qwen Code — see `docs/coding-cli-setup/`) don't reach it directly at all now; they go through the gateway below, same as everything else.
+| What | Reachable from |
+|---|---|
+| Gateway API | LAN, and the internet via the reverse proxy |
+| Gateway dashboard | LAN only |
+| Backends | Stack network only (through the gateway) |
+| qdrant | The server itself only |
 
-If you later want external CLI access too, that's a deliberate scope change — see the map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/issues/1)) before doing it, since it changes the security posture.
+## Gateway
 
-## The AI gateway (OmniRoute) — `proxy-ai.home` / `proxy-ai.haylan.ch`
+- **API port (`OMNIROUTE_API_PORT`)**: published on the server's LAN address. Two reverse-proxy hosts point at it:
+  - `proxy-ai.home`: LAN only.
+  - `proxy-ai.haylan.ch`: internet, through the DMZ that already forwards to the reverse proxy, which also manages the TLS certificate.
 
-As of [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) (migrated from LiteLLM), the gateway is OmniRoute:
+  Every call needs an [API key](gateway.md#api-keys), so the reverse proxy adds no auth of its own.
+- **Dashboard port (`OMNIROUTE_DASHBOARD_PORT`)**: published on the LAN only. It's never registered in the reverse proxy and has no port-forward, so it isn't reachable from the internet. Keep it that way: the dashboard manages every API key.
 
-- **`proxy-ai.home`** and **`proxy-ai.haylan.ch`** both point only at `${OMNIROUTE_PORT:-4000}` — the API port. Set up as two NPM Proxy Hosts pointing at this machine's LAN IP on that port; `proxy-ai.home` internal-only, `proxy-ai.haylan.ch` external via the DMZ already forwarding to NPM (let NPM issue/manage the TLS cert as usual).
+## Backends
-- The **dashboard** (`${OMNIROUTE_DASHBOARD_PORT:-20128}`) is never registered in NPM at all, and `docker-compose.yml` never publishes that port to the host either — it manages every workload's keys, so it doesn't belong on the public internet, same reasoning as LiteLLM's old `/ui`. Unlike LiteLLM, OmniRoute's split-port mode means this is structural (no network route exists) rather than an NPM path-deny rule that has to be maintained and could be misconfigured. Reach the dashboard only from the host itself or over SSH port-forward.
 
-**Every gateway call already requires a valid API key** (Bearer token, see `docs/proxy-key-onboarding.md`), so no extra NPM-level auth is needed for the external hostname.
+`llama-server` and `qwen-classifier` publish no host port (`expose` only). They have no authentication, so they're reachable only on the stack's Docker network, through the gateway. Making them directly reachable would put an unauthenticated inference endpoint on the network. That's a deliberate security change, not a config tweak.
 
-## RAG knowledge graph (Neo4j) — `knowledge.proxy-ai.home`
+## qdrant
 
-Set up as an NPM Proxy Host pointing at this machine's LAN IP on Neo4j's Browser port (`7474`, see `docker-compose.yml`'s `neo4j` service, [PR #50](https://git.arthurerlich.de/haylan/LLM-Server/pulls/50)). Internal-only, same as `proxy-ai.home` — no DMZ/external route, this is admin/dev tooling, not a client-facing endpoint. Bolt (`7687`, the actual query protocol) isn't proxied through NPM at all — clients on the LAN reach it directly at `<this-machine>:7687`.
+Bound to the server's loopback interface, because qdrant has no authentication. From another machine, use an SSH port-forward to the server.
-
-Qdrant's dashboard (`6333`) stays on its raw LAN IP/port for now — no hostname assigned yet.

docs/operations.md

@@ -0,0 +1,42 @@
+# Operations
+
+> **Source of truth:** [`scripts/update.sh`](../scripts/update.sh) and [`.env.example`](../.env.example). This page explains *what the script does and why*; the steps and defaults live in the source. If they disagree, the source wins.
+
+The stack runs on the **server**. After any change to the repo, run `./scripts/update.sh` there: it is the one command that brings the running stack in sync. It's safe to re-run at any time, and it only recreates what changed.
+
+## Self-update
+
+`git pull` runs before anything else. A script that updates itself isn't atomic against its own file changing mid-run. Pulling later once executed a mix of old and new script and compose content in a single pass. Keep the pull first.
+
+## Env sync
+
+`.env` is created from `.env.example` if it's missing. After that:
+
+- **Keys with a real default in `.env.example`** are tracked config, synced every run. A key missing from `.env` is added. A key whose value matches is left alone. A key whose value **differs** is a conflict:
+  - Interactively, all conflicts appear on one screen ([gum](#gum)). Selected keys take the new value, and unselected keys keep the server's value.
+  - Without a TTY (cron, CI, a pipe), any conflict is an error unless `--force` is passed, which accepts every new value.
+
+  To change a default for good, change `.env.example`, not the server's `.env`.
+- **Keys left blank in `.env.example`** ([secrets](#secrets) and [host-resolved values](#host-resolved-values)) are never synced. They're only filled when blank.
+- Keys removed from `.env.example` stay in the server's `.env`: the script adds keys and never prunes them. They're harmless, and you can delete them by hand.
+
+There's no rollback. If a bad config lands, `git revert` and re-run the script.
+
+## Secrets
+
+Random secrets are generated with `openssl` when blank and never overwritten after that. **`OMNIROUTE_STORAGE_ENCRYPTION_KEY` must never change after the first run**: it encrypts OmniRoute's whole database, and changing it makes the existing data unreadable. `OMNIROUTE_WS_BRIDGE_SECRET` is required by OmniRoute in production (its `docs/reference/ENVIRONMENT.md`). The per-workload API keys aren't here at all: see [gateway → API keys](gateway.md#api-keys).
+
+## Host-resolved values
+
+These are resolved on the server itself. Set them by hand only if resolution fails:
+
+- `SEARXNG_LAN_IP`, from `search.home`: see [gateway → Web search](gateway.md#web-search).
+- `HOST_VIDEO_GID` / `HOST_RENDER_GID`, from the host's `video` / `render` groups: see [architecture → GPU access](architecture.md#gpu-access).
+
+## Gum
+
+The conflict screen uses [gum](https://github.com/charmbracelet/gum), a single static binary. It's vendored as a release tarball under `scripts/vendor/`, because the server has no outbound internet access. Other architectures fall back to downloading it, and if that fails, to a plain-bash prompt. The extracted binary is cached under `.cache/gum/` (gitignored).
+
+## Model downloads
+
+Each model file has its own one-off downloader service in the `tools` compose profile. `update.sh` runs them every time. A `test -f` guard skips files that already exist, so re-runs don't re-download. Files live in the named `models` volume, not a host bind-mount. The downloaders run as root because the named volume is created root-owned. There's one service per model, so the small model's job never re-checks the large one.

docs/proxy-key-onboarding.md

@@ -1,26 +0,0 @@
-# Onboarding a workload onto the AI gateway
-
-How to issue a new per-workload API key against the OmniRoute gateway (see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) — the LiteLLM → OmniRoute migration; original gateway rationale in [issue #10](https://git.arthurerlich.de/haylan/LLM-Server/issues/10) / `docs/research/proxy-tool-choice.md`), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
-
-No workload in this stack itself needs a key right now — every client is external (a coding CLI, or another self-hosted service). There's no scripted mint yet either way: `POST /api/keys` needs a dashboard login session (`ManagementSessionAuth`), not a static bearer key like LiteLLM's old `/key/generate`, and that flow hasn't been verified against a live instance (see [issue #37](https://git.arthurerlich.de/haylan/LLM-Server/issues/37)). Create every key by hand for now, via the dashboard steps below.
-
-## Create the key
-
-1. Log into the omniroute dashboard. `DASHBOARD_PORT` (20128) is never published to the host (see `docker-compose.yml`'s `omniroute` service) — from the R9700 box itself, find the container's own address (`docker inspect -f '{{.NetworkSettings.Networks.ai_stack.IPAddress}}' omniroute`) and browse to `http://<that-ip>:20128` (the host can reach a container's bridge-network IP directly, published port or not). From elsewhere, SSH port-forward instead: `ssh -L 20128:<container-ip>:20128 <host>`, then browse `http://localhost:20128`.
-2. "Keys" → "Create API key".
-3. Label it `<workload>-<purpose>` — a short slug matching the workload, e.g. `paperless-ocr`, `gitea-code-review`, `claude-code-cli`. This label is the ledger: the dashboard lists keys by label, so there's no separate tracking doc to keep in sync.
-4. Copy the key value shown — it's only shown once at creation, per OmniRoute's docs.
-
-Once `POST /api/keys`'s session-auth flow is worked out (issue #37), the equivalent `curl` here can replace this manual step, the way `update.sh` used to automate LiteLLM's `/key/generate`.
-
-## Hand it to the workload
-
-Drop the key into that workload's own `.env` (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as `.env.example` in this repo for the existing stack.
-
-## Retiring or rotating a key
-
-No scheduled rotation. Revoke the key by hand in the dashboard ("Keys" → delete) only when:
-- the workload is retired, or
-- the key is suspected leaked/compromised.
-
-Then remove it from that workload's `.env`.

docs/research/README.md

@@ -0,0 +1,33 @@
+# Research index
+
+[← back to README](../../README.md)
+
+Each file here is a **dated snapshot**: research or a decision record as it stood when it was written. The files are never edited afterwards, and their status lives only in this table. A *superseded* doc is kept as history. Read it for *why* a choice was made, never for how the stack works today (for that, see [`architecture.md`](../architecture.md) and the source it links).
+
+| Doc | Status | Note |
+|---|---|---|
+| [500k-context-2-parallel-agents-gpu-roadmap.md](500k-context-2-parallel-agents-gpu-roadmap.md) | Background research | Hardware exploration for larger contexts. Not an active plan, and neither is [`../server-planing.md`](../server-planing.md), whatever this doc says about it |
+| [classifier-context-window-scaling.md](classifier-context-window-scaling.md) | Current | Classifier model sizing decision |
+| [colibri-ai-stack-evaluation.md](colibri-ai-stack-evaluation.md) | Current | Rejected option ("watch, don't integrate"). It lists ComfyUI, which has since been removed |
+| [connecting-cloud-providers.md](connecting-cloud-providers.md) | Superseded by the OmniRoute migration | Written for LiteLLM. It mentions Open WebUI, which has been removed |
+| [dashscope-websearch-selfhost-options.md](dashscope-websearch-selfhost-options.md) | Current | Why DashScope web search isn't emulated |
+| [fast-model-choice.md](fast-model-choice.md) | Current (drift) | Classifier model choice. Its addendum records the service name (`qwen-classifier`) and quant drift |
+| [langchain-pgvector-vs-litellm-pgvector.md](langchain-pgvector-vs-litellm-pgvector.md) | Superseded by the OmniRoute migration | LiteLLM-era RAG comparison |
+| [lazytainer-omniroute-idle-stop.md](lazytainer-omniroute-idle-stop.md) | Superseded: Lazytainer and `switch-model.sh` removed | Root cause of why Lazytainer's idle-stop never fired behind the gateway, which is the reason it was removed |
+| [litellm-knowledgebase.md](litellm-knowledgebase.md) | Superseded by the OmniRoute migration | LiteLLM vector-store feature. Its claim that Open WebUI is qdrant's only user no longer holds |
+| [litellm-searxng-search.md](litellm-searxng-search.md) | Superseded by [omniroute-qwen-websearch.md](omniroute-qwen-websearch.md) | Still the origin of the `search.home` DNS gotcha ([gateway → Web search](../gateway.md#web-search)) |
+| [omniroute-account-semaphore-timeout.md](omniroute-account-semaphore-timeout.md) | Current | Gateway timeout: per-connection semaphore |
+| [omniroute-direct-response-timeout-outage-2026-09-15.md](omniroute-direct-response-timeout-outage-2026-09-15.md) | Current | Gateway timeout: direct response headers, plus the ROCm stall incident |
+| [omniroute-memory-tool-injection-breaks-classifier.md](omniroute-memory-tool-injection-breaks-classifier.md) | Current | Memory tools must stay off for the classifier connection |
+| [omniroute-non-ping-sse-stream-timeout.md](omniroute-non-ping-sse-stream-timeout.md) | Current | Gateway timeout: first non-ping SSE event, and why `--cache-reuse` is set |
+| [omniroute-qwen-websearch.md](omniroute-qwen-websearch.md) | Current | Qwen Code routing and web search through the gateway |
+| [opencode-auto-compact.md](opencode-auto-compact.md) | Current | OpenCode's client-side compaction |
+| [opencode-cli-setup.md](opencode-cli-setup.md) | Current (drift) | The provider JSON shape is valid, but its base URL points at a backend directly. Use the gateway ([setup](../coding-cli-setup/opencode.md)) |
+| [ponytail-audit-2026-09-09.md](ponytail-audit-2026-09-09.md) | Superseded (point-in-time audit) | Findings are fixed or folded into the docs cleanup |
+| [proxy-request-priority.md](proxy-request-priority.md) | Superseded by the OmniRoute migration | Priority tiers via LiteLLM's scheduler, never built on OmniRoute |
+| [proxy-shadow-pricing.md](proxy-shadow-pricing.md) | Superseded by the OmniRoute migration | Shadow cost was wired into LiteLLM config, and nothing implements it now |
+| [proxy-tool-choice.md](proxy-tool-choice.md) | Superseded by the OmniRoute migration | Chose LiteLLM. It mentions Open WebUI, which has been removed |
+| [qwen3.8-27b-quant.md](qwen3.8-27b-quant.md) | Current | Main model quant and VRAM budget |
+| [qwen3.8-27b-tool-calling.md](qwen3.8-27b-tool-calling.md) | Current | Tool-calling parser risk for the main model |
+| [rocm-gpu-pin-and-render-group.md](rocm-gpu-pin-and-render-group.md) | Current (name drift) | Findings apply to `llama-server` + `qwen-classifier`. The service names in it are older |
+| [voidllm-evaluation.md](voidllm-evaluation.md) | Superseded by the OmniRoute migration | VoidLLM vs. LiteLLM, both moot |