haylanandClaude-Bot
cb9b3a9045
README.md and docs/proxy-key-onboarding.md were edited by later commits (b996b1f,24d749b) that the litellm-pgvector revert didn't touch, so they still named the now-removed MEMORY_RETRIEVAL_EMBEDDING_KEY var. Left docs/research/langchain-pgvector-vs-litellm-pgvector.md as-is — it's a dated record of the decision being reversed, not live config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
2.1 KiB
Onboarding a workload onto the AI proxy
How to issue a new per-workload API key against the LiteLLM proxy (see issue #10 / docs/research/proxy-tool-choice.md), so a new workload (a code-reviewer tool, Paperless-OCR, Gitea code review, etc.) gets its own key and its own visible usage/spend.
OPENWEBUI_LITELLM_KEY and LITELLM_PGVECTOR_EMBEDDING_KEY — the two keys this stack's own services need — are minted automatically by ./scripts/update.sh via the same API curl shows below; the steps here are for any other workload, or for those two if the automatic mint ever fails.
Create the key
- Log into LiteLLM's Admin UI (
/uion the proxy's deployed URL). - Create a new virtual key ("Keys" → "Create Key").
- Name it
<workload>-<purpose>— a short slug matching the workload, e.g.paperless-ocr,gitea-code-review,openwebui. This name is the ledger: LiteLLM's dashboard lists keys by name, so there's no separate tracking doc to keep in sync — name it clearly and the Usage tab tells you the rest (spend, last used, etc.). - Leave budget and rate limits unset (unlimited) by default. This is a shadow-cost estimate for fun, not real accounting or resource protection — see
docs/research/proxy-shadow-pricing.md. Only set a budget if a specific workload turns out to need a tripwire.
Or the same thing over the API (what update.sh does):
curl -sf -X POST "http://<proxy>:4000/key/generate" \
-H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
-H "Content-Type: application/json" \
-d '{"key_alias": "<workload>-<purpose>"}'
# -> {"key": "sk-...", ...}
Hand it to the workload
Drop the key into that workload's own .env (or equivalent config) — never into this repo. Each workload's config is the source of truth for its own credential, same pattern as .env.example in this repo for the existing stack.
Retiring or rotating a key
No scheduled rotation. Revoke the key by hand in the Admin UI ("Keys" → delete) only when:
- the workload is retired, or
- the key is suspected leaked/compromised.
Then remove it from that workload's .env.