haylanandClaude-Bot
472e3a4738
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37): replace the litellm/litellm-db services with omniroute, split-port mode (API_PORT published/reverse-proxied, DASHBOARD_PORT never published - tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as omniroute providers post-boot (no static config.yaml equivalent). No scripted per-workload key minting yet - omniroute's POST /api/keys needs a dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is a manual step for now (docs/proxy-key-onboarding.md). Caveat carried into the map and README: OmniRoute's own docs (docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md on its release/v3.8.51 branch) describe shipped features for AI-provider client-detection evasion, system-wide HTTPS interception via a locally installed root CA, and hiding credentials from secret scanners. Proceeding anyway was an explicit, informed user decision. Also drops the gateway-level memory/knowledgebase feature entirely (user: "I don't need it") - litellm-pgvector, pgvector-db, embedding-server, scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory- knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated and untouched. litellm-config.yaml deleted (was kept as a rollback reference, but there's no rollback path to a feature being deliberately removed). Not yet verified against real hardware - see issue #31's open tickets. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
2.2 KiB
Onboarding a workload onto the AI gateway
How to issue a new per-workload API key against the OmniRoute gateway (see issue #31 — the LiteLLM → OmniRoute migration; original gateway rationale in issue #10 / docs/research/proxy-tool-choice.md), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
OPENWEBUI_OMNIROUTE_KEY — the one key this stack's own services need — has no scripted mint yet: POST /api/keys needs a dashboard login session (ManagementSessionAuth), not a static bearer key like LiteLLM's old /key/generate, and that flow hasn't been verified against a live instance (see issue #37). Create every key — including that one — by hand for now, via the dashboard steps below.
Create the key
- Log into the omniroute dashboard (
http://<host>:${OMNIROUTE_DASHBOARD_PORT:-20128}— LAN/host-only, never published publicly, seedocker-compose.yml'somnirouteservice). - "Keys" → "Create API key".
- Label it
<workload>-<purpose>— a short slug matching the workload, e.g.paperless-ocr,gitea-code-review,openwebui. This label is the ledger: the dashboard lists keys by label, so there's no separate tracking doc to keep in sync. - Copy the key value shown — it's only shown once at creation, per OmniRoute's docs.
Once POST /api/keys's session-auth flow is worked out (issue #37), the equivalent curl here can replace this manual step, the way update.sh used to automate LiteLLM's /key/generate.
Hand it to the workload
Drop the key into that workload's own .env (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as .env.example in this repo for the existing stack.
Retiring or rotating a key
No scheduled rotation. Revoke the key by hand in the dashboard ("Keys" → delete) only when:
- the workload is retired, or
- the key is suspected leaked/compromised.
Then remove it from that workload's .env.