feat(gateway): migrate LiteLLM to OmniRoute, drop the memory/knowledgebase feature

LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37):
replace the litellm/litellm-db services with omniroute, split-port mode
(API_PORT published/reverse-proxied, DASHBOARD_PORT never published -
tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place
of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as
omniroute providers post-boot (no static config.yaml equivalent). No
scripted per-workload key minting yet - omniroute's POST /api/keys needs a
dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is
a manual step for now (docs/proxy-key-onboarding.md).

Caveat carried into the map and README: OmniRoute's own docs
(docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md
on its release/v3.8.51 branch) describe shipped features for AI-provider
client-detection evasion, system-wide HTTPS interception via a locally
installed root CA, and hiding credentials from secret scanners. Proceeding
anyway was an explicit, informed user decision.

Also drops the gateway-level memory/knowledgebase feature entirely (user:
"I don't need it") - litellm-pgvector, pgvector-db, embedding-server,
scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory-
knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated
and untouched. litellm-config.yaml deleted (was kept as a rollback
reference, but there's no rollback path to a feature being deliberately
removed).

Not yet verified against real hardware - see issue #31's open tickets.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
This commit is contained in:
2026-09-03 19:49:35 +02:00
co-authored by Claude-Bot
parent 4fe910a5f3
commit 472e3a4738
23 changed files with 133 additions and 1893 deletions
+26 -33
View File
@@ -30,48 +30,41 @@ LLAMA_CTX_SIZE=131072
# --- Open WebUI ---
WEBUI_PORT=8008
# Minted automatically by ./scripts/update.sh — leave blank. Manual fallback:
# docs/proxy-key-onboarding.md.
OPENWEBUI_LITELLM_KEY=
# No scripted mint yet — set OPENWEBUI_OMNIROUTE_KEY below by hand instead.
# --- Lazytainer ---
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.
LAZYTAINER_INACTIVE_TIMEOUT=900
# --- Embedding model (knowledgebase, see docs/memory-knowledgebase.md) ---
EMBEDDING_MODEL_FILE=nomic-embed-text-v1.5.Q8_0.gguf
# --- SearXNG web search (see docs/research/litellm-searxng-search.md) ---
# Resolved automatically by ./scripts/update.sh from search.home on this
# host — leave blank. Only set by hand if that resolution fails (e.g.
# search.home isn't a static DHCP reservation and its IP drifted).
SEARXNG_LAN_IP=
# --- LiteLLM proxy (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
LITELLM_PORT=4000
# --- OmniRoute gateway (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
# API_PORT is the only port published to the host/internet (reverse-proxied
# by NPM) — the dashboard (DASHBOARD_PORT) is never published, see
# docker-compose.yml's omniroute service comment.
OMNIROUTE_API_PORT=20129
OMNIROUTE_DASHBOARD_PORT=20128
# Random values, filled in automatically by ./scripts/update.sh — leave
# blank. LITELLM_SALT_KEY encrypts stored data; do not change it after the
# first run (existing encrypted data becomes unreadable if you do).
LITELLM_MASTER_KEY=
LITELLM_SALT_KEY=
LITELLM_DB_PASSWORD=
# Backs litellm's router state/rate-limits/budgets/cache invalidation
# (the redis service). Random value, filled in automatically — leave blank.
REDIS_PASSWORD=
# Admin UI login (https://<proxy>/ui). Without these, LiteLLM falls back to
# username "admin" / password = LITELLM_MASTER_KEY — set these instead so the
# master key never has to be typed into the browser. UI_PASSWORD is filled
# in automatically by ./scripts/update.sh if blank.
UI_USERNAME=admin
UI_PASSWORD=
# --- Knowledgebase (pgvector + litellm-pgvector, see docs/memory-knowledgebase.md) ---
# Random value, filled in automatically by ./scripts/update.sh — leave blank.
PGVECTOR_DB_PASSWORD=
# Auth key litellm-pgvector requires on its own API (its SERVER_API_KEY).
# Random value, filled in automatically by ./scripts/update.sh — leave blank.
LITELLM_PGVECTOR_API_KEY=
# A virtual key litellm-pgvector uses to call back into litellm for
# embeddings. Minted automatically by ./scripts/update.sh — leave blank.
# Manual fallback: docs/proxy-key-onboarding.md.
LITELLM_PGVECTOR_EMBEDDING_KEY=
# blank. Bootstrap dashboard admin password (log in at the dashboard port,
# change it there afterwards — this is only the first-boot value):
OMNIROUTE_INITIAL_PASSWORD=
# Signs dashboard session cookies:
OMNIROUTE_JWT_SECRET=
# Encrypts API key values at rest in omniroute's SQLite DB:
OMNIROUTE_API_KEY_SECRET=
# Encrypts the whole SQLite DB at rest. Do not change after first run —
# existing encrypted data becomes unreadable if you do (same caveat as
# LiteLLM's old LITELLM_SALT_KEY):
OMNIROUTE_STORAGE_ENCRYPTION_KEY=
# Per-deployment salts — random is fine, just needs to be stable:
OMNIROUTE_MACHINE_ID_SALT=
OMNIROUTE_CLI_SALT=
# Per-workload virtual keys — omniroute has no scripted /key/generate
# equivalent yet (its key-creation endpoint needs a dashboard login session,
# not a static bearer key — see issue #37), so mint these by hand in the
# dashboard for now. See docs/proxy-key-onboarding.md.
OPENWEBUI_OMNIROUTE_KEY=