Merge pull request 'Add Qdrant/Neo4j RAG storage + update.sh gum fixes' (#52) from feat-rag-databases into main

Reviewed-on: #52

.env.example

@@ -105,34 +105,6 @@ COMFYUI_PGID=
 HOST_VIDEO_GID=
 HOST_RENDER_GID=
 
-# --- llama.cpp / fast model (second, always-resident instance — see
-# docs/research/fast-model-choice.md and issue #44) ---
-# Qwen3-4B-Instruct-2507: architecturally non-thinking (never emits
-# <think> blocks, unlike Qwen3-1.7B/0.6B which need a per-call toggle) —
-# picked specifically so it stays fast enough for qwen-code's Auto Mode
-# classifier (Stage 1 wants ~300ms). Same publisher (unsloth) as the main
-# model for consistency.
-LLAMA_FAST_MODEL_FILE=Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf
-# Same reasoning as LLAMA_GPU_LAYERS above — full GPU offload, this model
-# is dense too.
-LLAMA_FAST_GPU_LAYERS=999
-# --ctx-size is the TOTAL across every LLAMA_FAST_PARALLEL slot, not per
-# request — same halving already called out for the main model above.
-# Was PARALLEL=2, silently halving this to 4096/slot — too small: a real
-# classifier call (hints + environment + recent tool-call history) hit
-# "exceeds the available context size (4096 tokens)" in practice, which
-# qwen-code surfaces as "Auto Mode couldn't classify this action
-# (Classifier stage 1 unavailable)" — see issue #5. Fixed by dropping to
-# a single slot instead of raising ctx-size (no extra VRAM, and this
-# service doesn't need concurrent classifier calls the way the main
-# model needs concurrent chat sessions) — the full 8192 now goes to the
-# one slot. If hints.allow/softDeny/hardDeny ever approach their
-# 50-entries-each ceiling, raise LLAMA_FAST_CTX_SIZE instead — qwen-code
-# caps those at 200 chars x 150 entries plus 40,000 chars of
-# historical-action context, which can exceed 8192 tokens worst-case.
-LLAMA_FAST_CTX_SIZE=8192
-LLAMA_FAST_PARALLEL=1
-
 # --- ComfyUI diffusion model (Qwen-Image, FP8 — see docs/research/
 # image-generation-model-choice.md and issue #42) ---
 # Three files: diffusion weights, text encoder, VAE — all from the official

README.md

@@ -14,7 +14,7 @@ See the wayfinder map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/
 
 llama.cpp's own API is internal-only — everything routes through the AI gateway below.
 
-Pointing Claude Code CLI, Kimi CLI, OpenCode CLI, or Qwen Code CLI at the local endpoint: see [`docs/coding-cli-setup/`](docs/coding-cli-setup/index.md).
+Pointing Claude Code CLI, Kimi CLI, or OpenCode CLI at the local endpoint: see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md).
 
 **Known risk**: Qwen3.8-27B's tool-calling reliability against llama.cpp's Anthropic shim is not yet verified (open upstream parser bugs against its model lineage) — see `docs/research/qwen3.8-27b-tool-calling.md`.
 
@@ -26,10 +26,15 @@ An [AI gateway/proxy](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) fr
 - Dashboard (key/provider management): LAN/host-only, never published to the internet — see `docs/network-access.md`.
 - Issuing a key for a new workload: [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md).
 
-Coding CLIs (see [`docs/coding-cli-setup/`](docs/coding-cli-setup/index.md)) route through the gateway — llama-server has no published host port. **Not yet verified**: none of this has been smoke-tested on real hardware yet — see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31)'s tickets for the open items (provider registration, per-workload key minting).
+Coding CLIs (see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md)) route through the gateway — llama-server has no published host port. **Not yet verified**: none of this has been smoke-tested on real hardware yet — see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31)'s tickets for the open items (provider registration, per-workload key minting).
 
 **Note on this choice**: OmniRoute's own docs (`docs/security/STEALTH_GUIDE.md`, `MITM-TPROXY-DECRYPT.md`, `PUBLIC_CREDS.md` in its repo) describe shipped features for evading AI-provider client detection, system-wide HTTPS interception via a locally-installed root CA, and hiding credentials from secret scanners. None of that is used by this stack's configuration, but it's a real characteristic of the upstream project — see issue #31's Notes for the full research trail before extending this integration further.
 
 ### Web search
 
 The gateway also fronts SearXNG-backed web search — see `docs/research/litellm-searxng-search.md` for the original research (still applicable — same standalone-endpoint pattern, see issue #31's #35).
+
+## What's not here
+
+- **Open WebUI** — this stack has no chat UI; every client is a coding CLI. Removed rather than kept idle.
+- **Gateway-level knowledgebase/memory** (`litellm-pgvector`, `pgvector-db`, a dedicated embedding model) — removed as unwanted, unrelated to OmniRoute's own lack of parity with it (see issue #31's #34). Superseded by OmniRoute's own built-in memory feature (opt-in via the dashboard, Settings → Memory): vector store is its bundled `sqlite-vec`, embeddings are a local ONNX model (Transformers.js, ~400MB, downloaded into the `omniroute-data` volume on first use) — no external services, no static config here.

docker-compose.yml

@@ -15,8 +15,7 @@ services:
     ipc: host
     # Caps this process's HIP hardware-queue allocation — works around
     # ROCm/ROCm#5706 (GPU pinned at 100%/boost-clock whenever two
-    # concurrent HIP contexts touch this card, confirmed on real hardware
-    # against llama-server-fast below). See the research doc above.
+    # concurrent HIP contexts touch this card). See the research doc above.
     environment:
       - GPU_MAX_HW_QUEUES=1
     volumes:
@@ -47,49 +46,6 @@ services:
       - "lazytainer.group.llamaserver.inactiveTimeout=${LAZYTAINER_INACTIVE_TIMEOUT:-900}"
       - "lazytainer.group.llamaserver.minPacketThreshold=2"
 
-  llama-server-fast:
-    image: ghcr.io/ggml-org/llama.cpp:server-rocm
-    container_name: llama-server-fast
-    devices:
-      - /dev/kfd
-      - /dev/dri
-    group_add:
-      - "${HOST_VIDEO_GID:?run scripts/update.sh first to resolve this}"
-      - "${HOST_RENDER_GID:?run scripts/update.sh first to resolve this}"
-    security_opt:
-      - seccomp=unconfined
-    ipc: host
-    # See llama-server's identical setting above — same fix, same bug.
-    environment:
-      - GPU_MAX_HW_QUEUES=1
-    volumes:
-      - models:/models
-    command: >
-      -m /models/${LLAMA_FAST_MODEL_FILE:-Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf}
-      --host 0.0.0.0
-      --port 8080
-      --n-gpu-layers ${LLAMA_FAST_GPU_LAYERS:-999}
-      --ctx-size ${LLAMA_FAST_CTX_SIZE:-8192}
-      --parallel ${LLAMA_FAST_PARALLEL:-1}
-      --flash-attn on
-      --cache-type-k q8_0
-      --cache-type-v q8_0
-      --reasoning off
-      --jinja
-    # Second, always-resident llama.cpp instance — small non-thinking model
-    # used as qwen-code's Auto Mode classifier fastModel, alongside the main
-    # 27B instance above. See docs/research/fast-model-choice.md and #44.
-    # Same ai-stack-only pattern as llama-server: no published host port.
-    expose:
-      - "8080"
-    restart: unless-stopped
-    networks: [ai-stack]
-    labels:
-      - "lazytainer.group.llamaserverfast.sleepMethod=stop"
-      - "lazytainer.group.llamaserverfast.ports=8080"
-      - "lazytainer.group.llamaserverfast.inactiveTimeout=${LAZYTAINER_INACTIVE_TIMEOUT:-900}"
-      - "lazytainer.group.llamaserverfast.minPacketThreshold=2"
-
   # ponytail: one-off downloader, not a standing service — run via
   # `docker compose --profile tools run --rm downloader`. Folded into
   # scripts/update.sh, which runs this every time; the `test -f` guard is
@@ -111,26 +67,9 @@ services:
         curl -L --fail --create-dirs -o /models/${LLAMA_MODEL_FILE:-Qwen3.8-27B-UD-Q4_K_XL.gguf}
         https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/${LLAMA_MODEL_FILE:-Qwen3.8-27B-UD-Q4_K_XL.gguf}
 
-  # Same test -f guard pattern as downloader above — fetches the second,
-  # smaller model for llama-server-fast. See issue #44.
-  downloader-fast:
-    image: curlimages/curl:latest
-    profiles: ["tools"]
-    user: root
-    volumes:
-      - models:/models
-    entrypoint: ["sh", "-c"]
-    command:
-      - >
-        test -f /models/${LLAMA_FAST_MODEL_FILE:-Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf} &&
-        echo "already downloaded, skipping" ||
-        curl -L --fail --create-dirs -o /models/${LLAMA_FAST_MODEL_FILE:-Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf}
-        https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF/resolve/main/${LLAMA_FAST_MODEL_FILE:-Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf}
-
   # Fetches the three Qwen-Image FP8 files ComfyUI needs (diffusion model,
-  # text encoder, VAE) — same test -f guard pattern as downloader/
-  # downloader-fast above. See docs/research/image-generation-model-choice.md
-  # and issue #42.
+  # text encoder, VAE) — same test -f guard pattern as downloader above.
+  # See docs/research/image-generation-model-choice.md and issue #42.
   #
   # ponytail: target paths assume ComfyUI's standard models/ layout under
   # BASE_STORAGE_PATH (/storage) — same "not independently confirmed against
@@ -184,7 +123,7 @@ services:
       # This image also wants GID env vars directly (its own README asks
       # for both these and group_add above) — same HOST_VIDEO_GID/
       # HOST_RENDER_GID resolved by scripts/update.sh, shared with
-      # llama-server/llama-server-fast now instead of comfyui-only vars.
+      # llama-server now instead of comfyui-only vars.
       - PUID=${COMFYUI_PUID}
       - PGID=${COMFYUI_PGID}
       - VIDEO_GID=${HOST_VIDEO_GID}
@@ -220,8 +159,6 @@ services:
     depends_on:
       llama-server:
         condition: service_started
-      llama-server-fast:
-        condition: service_started
     volumes:
       - omniroute-data:/app/data
     env_file: .env

docs/coding-cli-setup.md

@@ -0,0 +1,91 @@
+# Pointing a coding-agent CLI at this stack
+
+This stack routes through the [AI gateway](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) (OmniRoute, see issue #31) rather than talking to llama.cpp directly — llama.cpp's own port is internal-only now (see `docker-compose.yml`). The gateway exposes:
+
+- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1`
+- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_PORT:-4000}`
+
+Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, registered in the gateway (naming is yours to pick when adding the llama-cpp provider connection — this doc assumes `qwen3.8-27b-local` for continuity) — behind whichever wire format the client speaks.
+
+`<ai-box>` is this machine's LAN address, or `proxy-ai.home` if your local DNS resolves that hostname directly to the box — see `docs/network-access.md`. If you're running a coding CLI from this machine itself, `localhost` works too.
+
+**Each CLI needs its own virtual key** — create one per docs/proxy-key-onboarding.md (omniroute's dashboard, `<workload>-<purpose>` naming, e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli`). No budget set by default. These are the machine's interactive/high-priority workloads per `docs/proxy-request-priority.md`.
+
+> **Read this before relying on it for real work.** Qwen3.8-27B's tool-calling has **documented, open llama.cpp upstream bugs** (parser fails on text before `<tool_call>`, tool calls emitted as inert XML inside thinking blocks — see `docs/research/qwen3.8-27b-tool-calling.md`). Every setup below inherits this risk identically, regardless of which CLI or wire format you use. Don't trust it for unattended multi-step agentic work until you've run the smoke test in [issue #5](https://git.arthurerlich.de/haylan/LLM-Server/issues/5) (and the proxy-specific smoke test in [issue #17](https://git.arthurerlich.de/haylan/LLM-Server/issues/17)).
+
+## Claude Code CLI
+
+Claude Code speaks the **Anthropic Messages API** — point it at the gateway's unified endpoint, not llama.cpp directly:
+
+```bash
+export ANTHROPIC_BASE_URL=http://<ai-box>:${OMNIROUTE_PORT:-4000}
+export ANTHROPIC_API_KEY=<claude-code-cli virtual key>
+claude
+```
+
+Requires llama.cpp's `--jinja` flag (already set in `docker-compose.yml`) — without it, tool-use requests fail outright.
+
+## Kimi CLI
+
+Kimi CLI speaks plain **OpenAI Chat Completions**. Configure a provider block in its config file (`config.toml`):
+
+```toml
+[providers.openai]
+type = "openai"
+base_url = "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
+api_key = "<kimi-cli virtual key>"
+```
+
+If Kimi CLI's response parsing gets confused by Qwen's `<think>...</think>` reasoning tags, check its `reasoning_key` setting — it's configurable for non-standard local server responses.
+
+## OpenCode CLI
+
+Confirmed project: **`anomalyco/opencode`** (renamed from `sst/opencode` — don't confuse with the unrelated `opencode-ai/opencode` Go TUI). Docs: https://opencode.ai/docs/
+
+**Install**:
+```bash
+curl -fsSL https://opencode.ai/install | bash
+```
+
+**Config** (`opencode.json`, project root or `~/.config/opencode/opencode.json`):
+
+```json
+{
+  "$schema": "https://opencode.ai/config.json",
+  "provider": {
+    "aiproxy": {
+      "npm": "@ai-sdk/openai-compatible",
+      "name": "AI proxy (local)",
+      "options": {
+        "baseURL": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
+        "apiKey": "<opencode-cli virtual key>"
+      },
+      "models": {
+        "qwen3.8-27b-local": {
+          "name": "Qwen3.8-27B",
+          "limit": { "context": 65536, "output": 8192 }
+        }
+      }
+    }
+  }
+}
+```
+
+Set `limit.context` to match whatever `LLAMA_CTX_SIZE` this stack is actually running with (`.env`), not a value assumed from the model card — OpenCode uses it for its own context-management bookkeeping, not the server.
+
+Select the model with `aiproxy/qwen3.8-27b-local`.
+
+**OpenCode-specific risks** (on top of the shared Qwen3.8-27B tool-calling risk above):
+- Requires llama.cpp's `--jinja` flag (already set) — without it, OpenCode's unconditional tool-call scaffolding gets a 500.
+- [anomalyco/opencode#20669](https://github.com/anomalyco/opencode/issues/20669) (closed as "not planned" — a live, unfixed risk): OpenCode's `bash` tool crashes if the model omits the optional `description` field on a tool call; some local backends return `finish_reason: tool_calls` with an empty array, which can hang the agent loop instead of stopping cleanly.
+- Thinking-mode handling (`options.reasoningEffort`) is undocumented for models that emit inline `<think>` tags rather than a native reasoning API field — expect no effect from that config on this model; untested.
+
+## Summary
+
+| CLI | Wire format | Endpoint | Config |
+|---|---|---|---|
+| Claude Code | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_PORT:-4000}` | `ANTHROPIC_BASE_URL` env var |
+| Kimi CLI | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `config.toml` provider block |
+| OpenCode | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `opencode.json` provider block |
+
+Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/research/opencode-cli-setup.md`, `docs/proxy-key-onboarding.md`.

docs/coding-cli-setup/claude-code.md

@@ -1,13 +0,0 @@
-# Claude Code CLI
-
-[← back to overview](index.md)
-
-Claude Code speaks the **Anthropic Messages API** — point it at the gateway's unified endpoint, not llama.cpp directly:
-
-```bash
-export ANTHROPIC_BASE_URL=http://<ai-box>:${OMNIROUTE_PORT:-4000}
-export ANTHROPIC_API_KEY=<claude-code-cli virtual key>
-claude
-```
-
-Requires llama.cpp's `--jinja` flag (already set in `docker-compose.yml`) — without it, tool-use requests fail outright.

docs/coding-cli-setup/index.md

@@ -1,34 +0,0 @@
-# Pointing a coding-agent CLI at this stack
-
-[← back to README](../../README.md)
-
-This stack routes through the [AI gateway](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) (OmniRoute, see issue #31) rather than talking to llama.cpp directly — llama.cpp's own port is internal-only now (see `docker-compose.yml`). The gateway exposes:
-
-- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1`
-- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_PORT:-4000}`
-
-Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, registered in the gateway (naming is yours to pick when adding the llama-cpp provider connection — these docs assume `qwen3.8-27b-local` for continuity) — behind whichever wire format the client speaks.
-
-`<ai-box>` is this machine's LAN address, or `proxy-ai.home` if your local DNS resolves that hostname directly to the box — see `docs/network-access.md`. If you're running a coding CLI from this machine itself, `localhost` works too.
-
-**Each CLI needs its own virtual key** — create one per docs/proxy-key-onboarding.md (omniroute's dashboard, `<workload>-<purpose>` naming, e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli`). No budget set by default. These are the machine's interactive/high-priority workloads per `docs/proxy-request-priority.md`.
-
-> **Read this before relying on it for real work.** Qwen3.8-27B's tool-calling has **documented, open llama.cpp upstream bugs** (parser fails on text before `<tool_call>`, tool calls emitted as inert XML inside thinking blocks — see `docs/research/qwen3.8-27b-tool-calling.md`). Every CLI below inherits this risk identically, regardless of wire format. Don't trust it for unattended multi-step agentic work until you've run the smoke test in [issue #5](https://git.arthurerlich.de/haylan/LLM-Server/issues/5) (and the proxy-specific smoke test in [issue #17](https://git.arthurerlich.de/haylan/LLM-Server/issues/17)).
-
-## Per-CLI setup
-
-- [Claude Code CLI](claude-code.md)
-- [Kimi CLI](kimi-cli.md)
-- [OpenCode CLI](opencode.md)
-- [Qwen Code CLI](qwen-code.md)
-
-## Summary
-
-| CLI | Wire format | Endpoint | Config |
-|---|---|---|---|
-| [Claude Code](claude-code.md) | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_PORT:-4000}` | `ANTHROPIC_BASE_URL` env var |
-| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `config.toml` provider block |
-| [OpenCode](opencode.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `opencode.json` provider block |
-| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (2 models: chat + `fastModel`) | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `~/.qwen/settings.json` `modelProviders.openai` |
-
-Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/proxy-key-onboarding.md`.

docs/coding-cli-setup/kimi-cli.md

@@ -1,14 +0,0 @@
-# Kimi CLI
-
-[← back to overview](index.md)
-
-Kimi CLI speaks plain **OpenAI Chat Completions**. Configure a provider block in its config file (`config.toml`):
-
-```toml
-[providers.openai]
-type = "openai"
-base_url = "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
-api_key = "<kimi-cli virtual key>"
-```
-
-If Kimi CLI's response parsing gets confused by Qwen's `<think>...</think>` reasoning tags, check its `reasoning_key` setting — it's configurable for non-standard local server responses.

docs/coding-cli-setup/opencode.md

@@ -1,45 +0,0 @@
-# OpenCode CLI
-
-[← back to overview](index.md)
-
-Confirmed project: **`anomalyco/opencode`** (renamed from `sst/opencode` — don't confuse with the unrelated `opencode-ai/opencode` Go TUI). Docs: https://opencode.ai/docs/
-
-**Install**:
-```bash
-curl -fsSL https://opencode.ai/install | bash
-```
-
-**Config** (`opencode.json`, project root or `~/.config/opencode/opencode.json`):
-
-```json
-{
-  "$schema": "https://opencode.ai/config.json",
-  "provider": {
-    "aiproxy": {
-      "npm": "@ai-sdk/openai-compatible",
-      "name": "AI proxy (local)",
-      "options": {
-        "baseURL": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
-        "apiKey": "<opencode-cli virtual key>"
-      },
-      "models": {
-        "qwen3.8-27b-local": {
-          "name": "Qwen3.8-27B",
-          "limit": { "context": 65536, "output": 8192 }
-        }
-      }
-    }
-  }
-}
-```
-
-Set `limit.context` to the *per-slot* context this stack actually serves — `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from `.env` (262144 / 2 = 131072 by default), not raw `LLAMA_CTX_SIZE` and not a value assumed from the model card: llama.cpp divides `--ctx-size` across concurrent slots, so each request only gets one slot's share. OpenCode uses this for its own context-management bookkeeping, not the server.
-
-Select the model with `aiproxy/qwen3.8-27b-local`.
-
-**OpenCode-specific risks** (on top of the shared Qwen3.8-27B tool-calling risk — see [overview](index.md)):
-- Requires llama.cpp's `--jinja` flag (already set) — without it, OpenCode's unconditional tool-call scaffolding gets a 500.
-- [anomalyco/opencode#20669](https://github.com/anomalyco/opencode/issues/20669) (closed as "not planned" — a live, unfixed risk): OpenCode's `bash` tool crashes if the model omits the optional `description` field on a tool call; some local backends return `finish_reason: tool_calls` with an empty array, which can hang the agent loop instead of stopping cleanly.
-- Thinking-mode handling (`options.reasoningEffort`) is undocumented for models that emit inline `<think>` tags rather than a native reasoning API field — expect no effect from that config on this model; untested.
-
-Further reading: `docs/research/opencode-cli-setup.md`.

docs/coding-cli-setup/qwen-code.md

@@ -1,116 +0,0 @@
-# Qwen Code CLI
-
-[← back to overview](index.md)
-
-Qwen Code speaks plain **OpenAI Chat Completions**, and — unlike the other CLIs — needs *two* models: the main chat model, and a `fastModel` for Auto Mode's action classifier (a separate, always-resident, always-fast instance so classification doesn't queue behind chat prefill; see `docker-compose.yml`'s `llama-server-fast` service and `docs/research/fast-model-choice.md`). Both are registered as separate providers in OmniRoute but reachable through the same gateway URL. Config lives in `~/.qwen/settings.json`:
-
-```json
-{
-  "modelProviders": {
-    "openai": [
-      {
-        "id": "<main-model-provider-id-in-omniroute>",
-        "name": "qwen3.8-27b-local",
-        "envKey": "OMNIROUTE_API_KEY",
-        "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
-        "generationConfig": { "contextWindowSize": 131072 }
-      },
-      {
-        "id": "<fast-model-provider-id-in-omniroute>",
-        "name": "qwen3.8-27b-classifier",
-        "envKey": "OMNIROUTE_API_KEY",
-        "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
-        "generationConfig": {
-          "contextWindowSize": 8192,
-          "extra_body": { "chat_template_kwargs": { "enable_thinking": false } }
-        }
-      }
-    ]
-  },
-  "security": { "auth": { "selectedType": "openai" } },
-  "model": {
-    "name": "<main-model-provider-id-in-omniroute>",
-    "baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
-  },
-  "fastModel": "<fast-model-provider-id-in-omniroute>"
-}
-```
-
-- `envKey` names the environment variable Qwen Code reads the virtual key from — set `OMNIROUTE_API_KEY=<qwen-code-cli virtual key>` before launching. Both providers can share one virtual key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls.
-- **`contextWindowSize` is per-slot, not `LLAMA_CTX_SIZE` itself** — llama.cpp divides `--ctx-size` across `LLAMA_PARALLEL` concurrent slots, and each request only gets one slot's share (same correction applies to OpenCode's `limit.context`). Compute it per model from `.env`:
-  - Main model: `LLAMA_CTX_SIZE / LLAMA_PARALLEL` = `262144 / 2` = **131072**.
-  - Fast model: `LLAMA_FAST_CTX_SIZE / LLAMA_FAST_PARALLEL` = `8192 / 1` = **8192**. Undersizing this one specifically breaks Auto Mode ("Classifier stage 1 unavailable") once `hints.allow`/`softDeny`/`hardDeny` entries and recent-action history push a classifier call past it — see the `LLAMA_FAST_CTX_SIZE` comment in `.env.example` before raising it instead of `LLAMA_FAST_PARALLEL`.
-- `enable_thinking: false` on the fast model matters: the fast model file (`Qwen3-4B-Instruct-2507`) is already non-thinking, but this also suppresses `<think>` output on any fast-model swap that isn't, keeping classifier responses parseable.
-- Qwen Code also recognizes `advisorModel`, `visionModel`, `compactionModel`, `imageModel` for other model roles — none are wired up in this stack; only `fastModel` is required.
-
-## Web search via OmniRoute
-
-Qwen Code's own built-in web search (`tools.webSearch.enabled`) has nothing to search with here — leave it `false`. Instead this stack's SearXNG-backed search (README §"Web search") is exposed through a thin stdio MCP wrapper around OmniRoute's `/v1/search` REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. `~/.qwen/mcp-servers/omniroute-search/index.mjs` (needs `@modelcontextprotocol/sdk` and `zod`: `npm init -y && npm i @modelcontextprotocol/sdk zod` in that directory):
-
-```js
-import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
-import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
-import { z } from "zod";
-
-const BASE_URL = process.env.OMNIROUTE_BASE_URL || "http://proxy-ai.home";
-const API_KEY = process.env.OMNIROUTE_API_KEY;
-
-if (!API_KEY) {
-  console.error("OMNIROUTE_API_KEY is not set in the environment.");
-  process.exit(1);
-}
-
-const server = new McpServer({ name: "omniroute-search", version: "1.0.0" });
-
-server.registerTool(
-  "search",
-  {
-    description: "Web/news search via OmniRoute's /v1/search endpoint.",
-    inputSchema: { query: z.string().describe("Search query") },
-  },
-  async ({ query }) => {
-    const res = await fetch(`${BASE_URL}/v1/search`, {
-      method: "POST",
-      headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` },
-      body: JSON.stringify({ query }),
-    });
-    const text = await res.text();
-    if (!res.ok) return { content: [{ type: "text", text: `HTTP ${res.status}: ${text}` }], isError: true };
-    return { content: [{ type: "text", text }] };
-  }
-);
-
-await server.connect(new StdioServerTransport());
-```
-
-Register it in `~/.qwen/settings.json`:
-
-```json
-{
-  "mcpServers": {
-    "omniroute-search": { "command": "node", "args": ["<path-to>/index.mjs"] }
-  },
-  "tools": { "webSearch": { "enabled": false } }
-}
-```
-
-It reuses the same `OMNIROUTE_API_KEY` env var as the model providers above — the virtual key needs search permission in OmniRoute, not just chat-completions.
-
-## Auto Mode tuning
-
-Auto Mode's action classifier calls the fast model above — its own request can queue behind other stack traffic before the fast llama-server instance is warm, so the default classifier timeout is worth raising. And since this stack is a single trusted local proxy, it's reasonable to pre-approve requests to it rather than confirm every call:
-
-```json
-{
-  "permissions": {
-    "autoMode": {
-      "classifier": { "timeouts": { "stage1Ms": 600000 } },
-      "hints": { "allow": ["Requests to proxy-ai.home, my own local omniroute model proxy"] }
-    }
-  }
-}
-```
-
-`hints.allow` entries are free-text descriptions the classifier matches against, not exact strings — capped at 150 entries/200 chars each (see the `LLAMA_FAST_CTX_SIZE` note above for why that ceiling matters).
-
-Everything else in `~/.qwen/settings.json` (`hooks`, `security.auth`'s underlying tooling, editor prefs) is per-machine, not part of pointing at this stack — don't copy it wholesale between machines.

docs/network-access.md

@@ -4,7 +4,7 @@ This stack has no chat UI — every client is a coding CLI reaching the AI gatew
 
 ## llama.cpp's raw API stays LAN-only — deliberately
 
-The inference API (port `${LLAMA_PORT:-8080}`) is **not** registered in NPM and is **not** reachable externally. It has no authentication of its own — putting it on the public internet would mean an unauthenticated inference endpoint. Coding-agent CLIs (Claude Code, Kimi, OpenCode, Qwen Code — see `docs/coding-cli-setup/`) don't reach it directly at all now; they go through the gateway below, same as everything else.
+The inference API (port `${LLAMA_PORT:-8080}`) is **not** registered in NPM and is **not** reachable externally. It has no authentication of its own — putting it on the public internet would mean an unauthenticated inference endpoint. Coding-agent CLIs (Claude Code, Kimi, OpenCode — see `docs/coding-cli-setup.md`) don't reach it directly at all now; they go through the gateway below, same as everything else.
 
 If you later want external CLI access too, that's a deliberate scope change — see the map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/issues/1)) before doing it, since it changes the security posture.
 

scripts/switch-model.sh

@@ -11,12 +11,6 @@
 # it never reliably sleeps a service on its own. A scripted swap always
 # knows which service should go up/down, so it doesn't need that heuristic.
 #
-# llama-server-fast (the small classifier model, issue #44) is NOT part of
-# this swap — it's meant to stay always-resident. Worst case with comfyui up
-# is comfyui (~25GB, Qwen-Image FP8) + llama-server-fast (~5GB) ≈ 30GB,
-# still under the 32GB card but tight — unverified on real hardware, check
-# `docker compose ps` / VRAM usage after the first real swap.
-#
 # OmniRoute may show the just-stopped provider as errored/offline in its
 # dashboard for up to CREDENTIAL_HEALTH_CHECK_INTERVAL (default 5 min) after
 # a swap — cosmetic, not a functional problem (see the research doc above).

scripts/update.sh

@@ -93,12 +93,10 @@ conflict_old=()
 conflict_new=()
 while IFS='=' read -r key value; do
   [ -n "$value" ] || continue
+  current="$(grep -E "^${key}=" .env | head -1 | cut -d= -f2-)"
   if ! grep -qE "^${key}=" .env; then
     echo "${key}=${value}" >> .env
-    continue
-  fi
-  current="$(grep -E "^${key}=" .env | head -1 | cut -d= -f2-)"
-  if [ "$current" != "$value" ]; then
+  elif [ "$current" != "$value" ]; then
     conflict_keys+=("$key")
     conflict_old+=("$current")
     conflict_new+=("$value")