haylanandClaude-Bot
472e3a4738
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37): replace the litellm/litellm-db services with omniroute, split-port mode (API_PORT published/reverse-proxied, DASHBOARD_PORT never published - tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as omniroute providers post-boot (no static config.yaml equivalent). No scripted per-workload key minting yet - omniroute's POST /api/keys needs a dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is a manual step for now (docs/proxy-key-onboarding.md). Caveat carried into the map and README: OmniRoute's own docs (docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md on its release/v3.8.51 branch) describe shipped features for AI-provider client-detection evasion, system-wide HTTPS interception via a locally installed root CA, and hiding credentials from secret scanners. Proceeding anyway was an explicit, informed user decision. Also drops the gateway-level memory/knowledgebase feature entirely (user: "I don't need it") - litellm-pgvector, pgvector-db, embedding-server, scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory- knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated and untouched. litellm-config.yaml deleted (was kept as a rollback reference, but there's no rollback path to a feature being deliberately removed). Not yet verified against real hardware - see issue #31's open tickets. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
2.8 KiB
Network access: ai.home / ai.haylan.ch
Open WebUI is meant to be reachable as ai.home on the LAN and ai.haylan.ch from outside. This stack doesn't run its own reverse proxy — it publishes Open WebUI's port to the host (${WEBUI_PORT:-3000}, see docker-compose.yml) and relies on the existing Nginx Proxy Manager (NPM) instance already fronting other self-hosted services on this network.
What to set up in NPM
Two Proxy Hosts, both pointing at this machine's LAN IP on port ${WEBUI_PORT:-3000} (Open WebUI):
ai.home— internal only, no external DNS/TLS needed unless you want it.ai.haylan.ch— external, reachable via the DMZ already forwarding it to NPM; let NPM issue/manage the TLS cert as it does for other services.
llama.cpp's raw API stays LAN-only — deliberately
The inference API (port ${LLAMA_PORT:-8080}) is not registered in NPM and is not reachable via ai.haylan.ch. It has no authentication of its own (unlike Open WebUI, which has login enabled) — putting it on the public internet would mean an unauthenticated inference endpoint. Coding-agent CLIs (Claude Code, Kimi, OpenCode — see docs/coding-cli-setup.md) reach it directly over the LAN, using this machine's LAN IP or ai.home if your local DNS resolves that hostname straight to the box (bypassing NPM, which only fronts ports 80/443).
If you later want external CLI access too, that's a deliberate scope change — see the map (issue #1) before doing it, since it changes the security posture (the raw API would need its own auth in front of it).
The AI gateway (OmniRoute) — proxy.ai.home / proxy.ai.haylan.ch
As of issue #31 (migrated from LiteLLM), the gateway is OmniRoute — same NPM pattern as Open WebUI above, but a cleaner split than LiteLLM's ever was:
proxy.ai.homeandproxy.ai.haylan.chboth point only at${OMNIROUTE_API_PORT:-20129}— the API port.- The dashboard (
${OMNIROUTE_DASHBOARD_PORT:-20128}) is never registered in NPM at all, anddocker-compose.ymlnever publishes that port to the host either — it manages every workload's keys, so it doesn't belong on the public internet, same reasoning as LiteLLM's old/ui. Unlike LiteLLM, OmniRoute's split-port mode means this is structural (no network route exists) rather than an NPM path-deny rule that has to be maintained and could be misconfigured. Reach the dashboard only from the host itself or over SSH port-forward.
Every gateway call already requires a valid API key (Bearer token, see docs/proxy-key-onboarding.md) — the same bar Open WebUI clears with its own login — so no extra NPM-level auth is needed for the external hostname.