Author LiteLLM AI proxy service (resolves #14)

Adds litellm + litellm-db to docker-compose.yml, litellm-config.yaml with
custom shadow-cost pricing (Claude Sonnet 5 reference, per #11) and a
priority-scheduling stub (per #16, needs real-hardware smoke test), and
required LITELLM_MASTER_KEY/SALT_KEY/DB_PASSWORD env vars. Untested on real
hardware — that's #17. Open WebUI/coding CLIs still talk to llama.cpp
directly, migration is #15.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-25 07:11:15 +02:00
co-authored by Claude-Bot
parent 0342090e1c
commit 0aefb36a48
4 changed files with 77 additions and 0 deletions
+9
View File
@@ -19,3 +19,12 @@ OPENAI_API_KEY=local
# --- Lazytainer ---
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.
LAZYTAINER_INACTIVE_TIMEOUT=900
# --- LiteLLM proxy (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
LITELLM_PORT=4000
# Required — generate real random values before first run, e.g. `openssl rand -hex 32`.
# LITELLM_SALT_KEY encrypts stored data; do not change it after the first run
# (existing encrypted data becomes unreadable if you do).
LITELLM_MASTER_KEY=
LITELLM_SALT_KEY=
LITELLM_DB_PASSWORD=litellm