haylanandClaude-Bot
472e3a4738
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37): replace the litellm/litellm-db services with omniroute, split-port mode (API_PORT published/reverse-proxied, DASHBOARD_PORT never published - tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as omniroute providers post-boot (no static config.yaml equivalent). No scripted per-workload key minting yet - omniroute's POST /api/keys needs a dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is a manual step for now (docs/proxy-key-onboarding.md). Caveat carried into the map and README: OmniRoute's own docs (docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md on its release/v3.8.51 branch) describe shipped features for AI-provider client-detection evasion, system-wide HTTPS interception via a locally installed root CA, and hiding credentials from secret scanners. Proceeding anyway was an explicit, informed user decision. Also drops the gateway-level memory/knowledgebase feature entirely (user: "I don't need it") - litellm-pgvector, pgvector-db, embedding-server, scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory- knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated and untouched. litellm-config.yaml deleted (was kept as a rollback reference, but there's no rollback path to a feature being deliberately removed). Not yet verified against real hardware - see issue #31's open tickets. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
27 lines
2.2 KiB
Markdown
27 lines
2.2 KiB
Markdown
# Onboarding a workload onto the AI gateway
|
|
|
|
How to issue a new per-workload API key against the OmniRoute gateway (see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) — the LiteLLM → OmniRoute migration; original gateway rationale in [issue #10](https://git.arthurerlich.de/haylan/LLM-Server/issues/10) / `docs/research/proxy-tool-choice.md`), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
|
|
|
|
`OPENWEBUI_OMNIROUTE_KEY` — the one key this stack's own services need — has no scripted mint yet: `POST /api/keys` needs a dashboard login session (`ManagementSessionAuth`), not a static bearer key like LiteLLM's old `/key/generate`, and that flow hasn't been verified against a live instance (see [issue #37](https://git.arthurerlich.de/haylan/LLM-Server/issues/37)). Create every key — including that one — by hand for now, via the dashboard steps below.
|
|
|
|
## Create the key
|
|
|
|
1. Log into the omniroute dashboard (`http://<host>:${OMNIROUTE_DASHBOARD_PORT:-20128}` — LAN/host-only, never published publicly, see `docker-compose.yml`'s `omniroute` service).
|
|
2. "Keys" → "Create API key".
|
|
3. Label it `<workload>-<purpose>` — a short slug matching the workload, e.g. `paperless-ocr`, `gitea-code-review`, `openwebui`. This label is the ledger: the dashboard lists keys by label, so there's no separate tracking doc to keep in sync.
|
|
4. Copy the key value shown — it's only shown once at creation, per OmniRoute's docs.
|
|
|
|
Once `POST /api/keys`'s session-auth flow is worked out (issue #37), the equivalent `curl` here can replace this manual step, the way `update.sh` used to automate LiteLLM's `/key/generate`.
|
|
|
|
## Hand it to the workload
|
|
|
|
Drop the key into that workload's own `.env` (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as `.env.example` in this repo for the existing stack.
|
|
|
|
## Retiring or rotating a key
|
|
|
|
No scheduled rotation. Revoke the key by hand in the dashboard ("Keys" → delete) only when:
|
|
- the workload is retired, or
|
|
- the key is suspected leaked/compromised.
|
|
|
|
Then remove it from that workload's `.env`.
|