haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gateway): migrate LiteLLM to OmniRoute, drop the memory/knowledgebase feature

Browse Source 
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37):
replace the litellm/litellm-db services with omniroute, split-port mode
(API_PORT published/reverse-proxied, DASHBOARD_PORT never published -
tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place
of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as
omniroute providers post-boot (no static config.yaml equivalent). No
scripted per-workload key minting yet - omniroute's POST /api/keys needs a
dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is
a manual step for now (docs/proxy-key-onboarding.md).

Caveat carried into the map and README: OmniRoute's own docs
(docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md
on its release/v3.8.51 branch) describe shipped features for AI-provider
client-detection evasion, system-wide HTTPS interception via a locally
installed root CA, and hiding credentials from secret scanners. Proceeding
anyway was an explicit, informed user decision.

Also drops the gateway-level memory/knowledgebase feature entirely (user:
"I don't need it") - litellm-pgvector, pgvector-db, embedding-server,
scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory-
knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated
and untouched. litellm-config.yaml deleted (was kept as a rollback
reference, but there's no rollback path to a feature being deliberately
removed).

Not yet verified against real hardware - see issue #31's open tickets.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
This commit is contained in:
haylan
2026-09-03 19:49:35 +02:00
co-authored by Claude-Bot
parent 4fe910a5f3
commit 472e3a4738
23 changed files with 133 additions and 1893 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+1 -1
View File
@@ -10,4 +10,4 @@ Single-context: `CONTEXT.md` + `docs/adr/` at the repo root. See `docs/agents/do
### Deploying changes
The running stack lives on a separate box (the R9700 server), not wherever this repo is being edited. After **any** change to `docker-compose.yml`, `litellm-config.yaml`, `.env.example`, or a script under `scripts/`, commit/push it, then run `./scripts/update.sh` on the server to apply it — don't just describe the change as done. If this session doesn't have shell access to the server, say so explicitly and tell the user to run it themselves rather than leaving it unsaid.
The running stack lives on a separate box (the R9700 server), not wherever this repo is being edited. After **any** change to `docker-compose.yml`, `.env.example`, or a script under `scripts/`, commit/push it, then run `./scripts/update.sh` on the server to apply it — don't just describe the change as done. If this session doesn't have shell access to the server, say so explicitly and tell the user to run it themselves rather than leaving it unsaid.