haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(stack): remove Open WebUI and Qdrant

Browse Source 
No longer needed - every client is a coding CLI behind the OmniRoute
gateway, not a chat UI. Drops the open-webui and qdrant services,
WEBUI_PORT/OPENWEBUI_OMNIROUTE_KEY env vars, and the openwebui-data/
qdrant-data volumes. Qdrant only ever served Open WebUI's own built-in
memory/RAG (unrelated to the gateway-level knowledgebase removed in
472e3a4), so it goes too rather than sit unused.

Docs updated: README, docs/network-access.md (ai.home/ai.haylan.ch
section was entirely about Open WebUI, rewritten around the gateway),
docs/proxy-key-onboarding.md, docs/proxy-request-priority.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
This commit is contained in:
haylan
2026-09-03 19:55:31 +02:00
co-authored by Claude-Bot
parent 472e3a4738
commit 3bbda098b3
7 changed files with 35 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+11 -7
View File
@@ -1,6 +1,6 @@
# LLM-Server
Local AI inference stack: llama.cpp (ROCm) serving Qwen3.8-27B on an AMD Radeon AI PRO R9700, fronted by Open WebUI (RAG + Memory via Qdrant), with Lazytainer auto-suspending the inference container when idle.
Local AI inference stack: llama.cpp (ROCm) serving Qwen3.8-27B on an AMD Radeon AI PRO R9700, fronted by the OmniRoute AI gateway, with Lazytainer auto-suspending the inference container when idle.
See the wayfinder map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/issues/1)) for the full architecture rationale and open questions.
@@ -10,10 +10,9 @@ See the wayfinder map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/
./scripts/update.sh
```
`update.sh` creates `.env` from `.env.example` if missing, fills in every random secret it can generate itself (via `openssl`, `SEARXNG_LAN_IP` resolved from `search.home` on this host), downloads the model GGUF into the `models` volume if it's not there yet, then pulls/builds/brings up the whole stack. Per-workload API keys (e.g. `OPENWEBUI_OMNIROUTE_KEY`) have no scripted mint yet — see [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md). Safe to re-run any time — it only fills in what's still blank, skips the model if already downloaded, and only recreates what changed.
`update.sh` creates `.env` from `.env.example` if missing, fills in every random secret it can generate itself (via `openssl`, `SEARXNG_LAN_IP` resolved from `search.home` on this host), downloads the model GGUF into the `models` volume if it's not there yet, then pulls/builds/brings up the whole stack. Safe to re-run any time — it only fills in what's still blank, skips the model if already downloaded, and only recreates what changed.
- Open WebUI: `http://<this-machine>:3000` locally, or `ai.home` / `ai.haylan.ch` once routed through Nginx Proxy Manager — see [`docs/network-access.md`](docs/network-access.md). First signup becomes the admin account (`WEBUI_AUTH` is on).
- llama.cpp's own API is internal-only now — everything routes through the AI proxy below.
llama.cpp's own API is internal-only — everything routes through the AI gateway below.
Pointing Claude Code CLI, Kimi CLI, or OpenCode CLI at the local endpoint: see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md).
@@ -27,12 +26,17 @@ An [AI gateway/proxy](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) fr
- Dashboard (key/provider management): LAN/host-only, never published to the internet — see `docs/network-access.md`.
- Issuing a key for a new workload: [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md).
Open WebUI and the coding CLIs (see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md)) route through the gateway — llama-server has no published host port. **Not yet verified**: none of this has been smoke-tested on real hardware yet — see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31)'s tickets for the open items (provider registration, per-workload key minting, Redis integration).
Coding CLIs (see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md)) route through the gateway — llama-server has no published host port. **Not yet verified**: none of this has been smoke-tested on real hardware yet — see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31)'s tickets for the open items (provider registration, per-workload key minting).
**Note on this choice**: OmniRoute's own docs (`docs/security/STEALTH_GUIDE.md`, `MITM-TPROXY-DECRYPT.md`, `PUBLIC_CREDS.md` in its repo) describe shipped features for evading AI-provider client detection, system-wide HTTPS interception via a locally-installed root CA, and hiding credentials from secret scanners. None of that is used by this stack's configuration, but it's a real characteristic of the upstream project — see issue #31's Notes for the full research trail before extending this integration further.
### Web search
The gateway also fronts SearXNG-backed web search, wired at the gateway layer so every client gets it, not just Open WebUI — see `docs/research/litellm-searxng-search.md` for the original research (still applicable — same standalone-endpoint pattern, see issue #31's #35).
The gateway also fronts SearXNG-backed web search — see `docs/research/litellm-searxng-search.md` for the original research (still applicable — same standalone-endpoint pattern, see issue #31's #35).
The gateway-level knowledgebase/memory feature (litellm-pgvector, pgvector-db, a dedicated embedding model) that used to sit alongside this was removed — this stack doesn't use it. Open WebUI's own built-in memory/RAG (backed by `qdrant`) is unrelated and unaffected.
## What's not here
Two features from earlier iterations of this stack were deliberately removed, not just left unfinished:
- **Open WebUI + Qdrant** — this stack has no chat UI; every client is a coding CLI. Removed rather than kept idle.
- **Gateway-level knowledgebase/memory** (`litellm-pgvector`, `pgvector-db`, a dedicated embedding model) — removed as unwanted, unrelated to OmniRoute's own lack of parity with it (see issue #31's #34).