23 lines
1.6 KiB
Markdown
23 lines
1.6 KiB
Markdown
# Onboarding a workload onto the AI proxy
|
|
|
|
How to issue a new per-workload API key against the LiteLLM proxy (see [issue #10](https://git.arthurerlich.de/haylan/LLM-Server/issues/10) / `docs/research/proxy-tool-choice.md`), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
|
|
|
|
## Create the key
|
|
|
|
1. Log into LiteLLM's Admin UI (`/ui` on the proxy's deployed URL).
|
|
2. Create a new virtual key ("Keys" → "Create Key").
|
|
3. Name it `<workload>-<purpose>` — a short slug matching the workload, e.g. `paperless-ocr`, `gitea-code-review`, `openwebui`. This name is the ledger: LiteLLM's dashboard lists keys by name, so there's no separate tracking doc to keep in sync — name it clearly and the Usage tab tells you the rest (spend, last used, etc.).
|
|
4. Leave budget and rate limits unset (unlimited) by default. This is a shadow-cost estimate for fun, not real accounting or resource protection — see `docs/research/proxy-shadow-pricing.md`. Only set a budget if a specific workload turns out to need a tripwire.
|
|
|
|
## Hand it to the workload
|
|
|
|
Drop the key into that workload's own `.env` (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as `.env.example` in this repo for the existing stack.
|
|
|
|
## Retiring or rotating a key
|
|
|
|
No scheduled rotation. Revoke the key by hand in the Admin UI ("Keys" → delete) only when:
|
|
- the workload is retired, or
|
|
- the key is suspected leaked/compromised.
|
|
|
|
Then remove it from that workload's `.env`.
|