feat(gateway): migrate LiteLLM to OmniRoute, drop the memory/knowledgebase feature
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37): replace the litellm/litellm-db services with omniroute, split-port mode (API_PORT published/reverse-proxied, DASHBOARD_PORT never published - tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as omniroute providers post-boot (no static config.yaml equivalent). No scripted per-workload key minting yet - omniroute's POST /api/keys needs a dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is a manual step for now (docs/proxy-key-onboarding.md). Caveat carried into the map and README: OmniRoute's own docs (docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md on its release/v3.8.51 branch) describe shipped features for AI-provider client-detection evasion, system-wide HTTPS interception via a locally installed root CA, and hiding credentials from secret scanners. Proceeding anyway was an explicit, informed user decision. Also drops the gateway-level memory/knowledgebase feature entirely (user: "I don't need it") - litellm-pgvector, pgvector-db, embedding-server, scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory- knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated and untouched. litellm-config.yaml deleted (was kept as a rollback reference, but there's no rollback path to a feature being deliberately removed). Not yet verified against real hardware - see issue #31's open tickets. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
This commit is contained in:
@@ -15,13 +15,11 @@ The inference API (port `${LLAMA_PORT:-8080}`) is **not** registered in NPM and
|
||||
|
||||
If you later want external CLI access too, that's a deliberate scope change — see the map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/issues/1)) before doing it, since it changes the security posture (the raw API would need its own auth in front of it).
|
||||
|
||||
## The AI proxy (LiteLLM) — `proxy.ai.home` / `proxy.ai.haylan.ch`
|
||||
## The AI gateway (OmniRoute) — `proxy.ai.home` / `proxy.ai.haylan.ch`
|
||||
|
||||
Once the gateway from [issue #9](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) is deployed, it gets its own hostnames, same NPM pattern as Open WebUI above:
|
||||
As of [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) (migrated from LiteLLM), the gateway is OmniRoute — same NPM pattern as Open WebUI above, but a cleaner split than LiteLLM's ever was:
|
||||
|
||||
- **`proxy.ai.home`** — internal only, fronts the full LiteLLM port (API + Admin UI).
|
||||
- **`proxy.ai.haylan.ch`** — external, via the DMZ/NPM. Fronts only the inference API paths.
|
||||
- **`proxy.ai.home`** and **`proxy.ai.haylan.ch`** both point only at `${OMNIROUTE_API_PORT:-20129}` — the API port.
|
||||
- The **dashboard** (`${OMNIROUTE_DASHBOARD_PORT:-20128}`) is never registered in NPM at all, and `docker-compose.yml` never publishes that port to the host either — it manages every workload's keys, so it doesn't belong on the public internet, same reasoning as LiteLLM's old `/ui`. Unlike LiteLLM, OmniRoute's split-port mode means this is structural (no network route exists) rather than an NPM path-deny rule that has to be maintained and could be misconfigured. Reach the dashboard only from the host itself or over SSH port-forward.
|
||||
|
||||
**Every proxy call already requires a valid virtual key** (Bearer token, see `docs/proxy-key-onboarding.md`) — the same bar Open WebUI clears with its own login — so no extra NPM-level auth is needed for the external hostname.
|
||||
|
||||
**LiteLLM's Admin UI (`/ui`) stays LAN-only**, same reasoning as llama.cpp's raw API: it manages every workload's keys and budgets, so it doesn't belong on the public internet. LiteLLM serves `/ui` on the same port as its API by default, so `proxy.ai.haylan.ch`'s NPM Proxy Host needs an explicit rule denying the `/ui` path (a "Deny" custom location, same UI as the "Advanced" tab used for other NPM hosts) — `proxy.ai.home` has no such restriction and reaches both the API and the Admin UI.
|
||||
**Every gateway call already requires a valid API key** (Bearer token, see `docs/proxy-key-onboarding.md`) — the same bar Open WebUI clears with its own login — so no extra NPM-level auth is needed for the external hostname.
|
||||
|
||||
Reference in New Issue
Block a user