haylanandClaude-Bot
cb9b3a9045
README.md and docs/proxy-key-onboarding.md were edited by later commits (b996b1f,24d749b) that the litellm-pgvector revert didn't touch, so they still named the now-removed MEMORY_RETRIEVAL_EMBEDDING_KEY var. Left docs/research/langchain-pgvector-vs-litellm-pgvector.md as-is — it's a dated record of the decision being reversed, not live config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
35 lines
2.1 KiB
Markdown
35 lines
2.1 KiB
Markdown
# Onboarding a workload onto the AI proxy
|
|
|
|
How to issue a new per-workload API key against the LiteLLM proxy (see [issue #10](https://git.arthurerlich.de/haylan/LLM-Server/issues/10) / `docs/research/proxy-tool-choice.md`), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
|
|
|
|
`OPENWEBUI_LITELLM_KEY` and `LITELLM_PGVECTOR_EMBEDDING_KEY` — the two keys this stack's own services need — are minted automatically by `./scripts/update.sh` via the same API `curl` shows below; the steps here are for any other workload, or for those two if the automatic mint ever fails.
|
|
|
|
## Create the key
|
|
|
|
1. Log into LiteLLM's Admin UI (`/ui` on the proxy's deployed URL).
|
|
2. Create a new virtual key ("Keys" → "Create Key").
|
|
3. Name it `<workload>-<purpose>` — a short slug matching the workload, e.g. `paperless-ocr`, `gitea-code-review`, `openwebui`. This name is the ledger: LiteLLM's dashboard lists keys by name, so there's no separate tracking doc to keep in sync — name it clearly and the Usage tab tells you the rest (spend, last used, etc.).
|
|
4. Leave budget and rate limits unset (unlimited) by default. This is a shadow-cost estimate for fun, not real accounting or resource protection — see `docs/research/proxy-shadow-pricing.md`. Only set a budget if a specific workload turns out to need a tripwire.
|
|
|
|
Or the same thing over the API (what `update.sh` does):
|
|
|
|
```bash
|
|
curl -sf -X POST "http://<proxy>:4000/key/generate" \
|
|
-H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"key_alias": "<workload>-<purpose>"}'
|
|
# -> {"key": "sk-...", ...}
|
|
```
|
|
|
|
## Hand it to the workload
|
|
|
|
Drop the key into that workload's own `.env` (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as `.env.example` in this repo for the existing stack.
|
|
|
|
## Retiring or rotating a key
|
|
|
|
No scheduled rotation. Revoke the key by hand in the Admin UI ("Keys" → delete) only when:
|
|
- the workload is retired, or
|
|
- the key is suspected leaked/compromised.
|
|
|
|
Then remove it from that workload's `.env`.
|