haylanandClaude-Bot
90ef1a1061
Not omniroute's own internal port (API_PORT stays at its default 20129, unreconfigured) - just the Docker port mapping, so existing NPM/firewall config pointed at :4000 keeps working without changes on that end. New OMNIROUTE_PORT env var is the host side of "OMNIROUTE_PORT:API_PORT" in docker-compose.yml's ports: entry. Also corrected docs/proxy-key-onboarding.md's dashboard-access instructions - DASHBOARD_PORT was never published to the host in the first place, so "http://<host>:20128" was never actually reachable as written; documented reaching it via the container's own bridge-network IP or an SSH port-forward instead. llama-server remains unexposed (no ports: entry, only expose:) - unaffected by this change, confirming it stays that way. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
27 lines
2.6 KiB
Markdown
27 lines
2.6 KiB
Markdown
# Onboarding a workload onto the AI gateway
|
|
|
|
How to issue a new per-workload API key against the OmniRoute gateway (see [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) — the LiteLLM → OmniRoute migration; original gateway rationale in [issue #10](https://git.arthurerlich.de/haylan/LLM-Server/issues/10) / `docs/research/proxy-tool-choice.md`), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
|
|
|
|
No workload in this stack itself needs a key right now — every client is external (a coding CLI, or another self-hosted service). There's no scripted mint yet either way: `POST /api/keys` needs a dashboard login session (`ManagementSessionAuth`), not a static bearer key like LiteLLM's old `/key/generate`, and that flow hasn't been verified against a live instance (see [issue #37](https://git.arthurerlich.de/haylan/LLM-Server/issues/37)). Create every key by hand for now, via the dashboard steps below.
|
|
|
|
## Create the key
|
|
|
|
1. Log into the omniroute dashboard. `DASHBOARD_PORT` (20128) is never published to the host (see `docker-compose.yml`'s `omniroute` service) — from the R9700 box itself, find the container's own address (`docker inspect -f '{{.NetworkSettings.Networks.ai_stack.IPAddress}}' omniroute`) and browse to `http://<that-ip>:20128` (the host can reach a container's bridge-network IP directly, published port or not). From elsewhere, SSH port-forward instead: `ssh -L 20128:<container-ip>:20128 <host>`, then browse `http://localhost:20128`.
|
|
2. "Keys" → "Create API key".
|
|
3. Label it `<workload>-<purpose>` — a short slug matching the workload, e.g. `paperless-ocr`, `gitea-code-review`, `claude-code-cli`. This label is the ledger: the dashboard lists keys by label, so there's no separate tracking doc to keep in sync.
|
|
4. Copy the key value shown — it's only shown once at creation, per OmniRoute's docs.
|
|
|
|
Once `POST /api/keys`'s session-auth flow is worked out (issue #37), the equivalent `curl` here can replace this manual step, the way `update.sh` used to automate LiteLLM's `/key/generate`.
|
|
|
|
## Hand it to the workload
|
|
|
|
Drop the key into that workload's own `.env` (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as `.env.example` in this repo for the existing stack.
|
|
|
|
## Retiring or rotating a key
|
|
|
|
No scheduled rotation. Revoke the key by hand in the dashboard ("Keys" → delete) only when:
|
|
- the workload is retired, or
|
|
- the key is suspected leaked/compromised.
|
|
|
|
Then remove it from that workload's `.env`.
|