feat(gateway): migrate LiteLLM to OmniRoute, drop the memory/knowledgebase feature

LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37):
replace the litellm/litellm-db services with omniroute, split-port mode
(API_PORT published/reverse-proxied, DASHBOARD_PORT never published -
tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place
of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as
omniroute providers post-boot (no static config.yaml equivalent). No
scripted per-workload key minting yet - omniroute's POST /api/keys needs a
dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is
a manual step for now (docs/proxy-key-onboarding.md).

Caveat carried into the map and README: OmniRoute's own docs
(docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md
on its release/v3.8.51 branch) describe shipped features for AI-provider
client-detection evasion, system-wide HTTPS interception via a locally
installed root CA, and hiding credentials from secret scanners. Proceeding
anyway was an explicit, informed user decision.

Also drops the gateway-level memory/knowledgebase feature entirely (user:
"I don't need it") - litellm-pgvector, pgvector-db, embedding-server,
scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory-
knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated
and untouched. litellm-config.yaml deleted (was kept as a rollback
reference, but there's no rollback path to a feature being deliberately
removed).

Not yet verified against real hardware - see issue #31's open tickets.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197

docker-compose.yml

@@ -35,36 +35,6 @@ services:
       - "lazytainer.group.llamaserver.inactiveTimeout=${LAZYTAINER_INACTIVE_TIMEOUT:-900}"
       - "lazytainer.group.llamaserver.minPacketThreshold=2"
 
-  embedding-server:
-    image: ghcr.io/ggml-org/llama.cpp:server-rocm
-    container_name: embedding-server
-    devices:
-      - /dev/kfd
-      - /dev/dri
-    group_add:
-      - video
-      - render
-    security_opt:
-      - seccomp=unconfined
-    ipc: host
-    volumes:
-      - models:/models
-    command: >
-      -m /models/${EMBEDDING_MODEL_FILE:-nomic-embed-text-v1.5.Q8_0.gguf}
-      --host 0.0.0.0
-      --port 8080
-      --embeddings
-      --pooling mean
-      --n-gpu-layers 999
-      --ctx-size 8192
-    # A dedicated embedding model — the chat model isn't embedding-trained
-    # and llama.cpp serves one model per process, so this is a second small
-    # instance, not a mode switch on llama-server. See
-    # docs/research/litellm-knowledgebase.md. Small enough (~150MB Q8) to
-    # run alongside the chat model's ~19.6GB in the R9700's 32GB VRAM.
-    restart: unless-stopped
-    networks: [ai-stack]
-
   # ponytail: one-off downloader, not a standing service — run via
   # `docker compose --profile tools run --rm downloader`. Folded into
   # scripts/update.sh, which runs this every time; the `test -f` guard is
@@ -86,23 +56,6 @@ services:
         curl -L --fail --create-dirs -o /models/${LLAMA_MODEL_FILE:-Qwen3.8-27B-UD-Q4_K_XL.gguf}
         https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/${LLAMA_MODEL_FILE:-Qwen3.8-27B-UD-Q4_K_XL.gguf}
 
-  # ponytail: same one-off pattern as `downloader`, for the embedding model —
-  # run via `docker compose --profile tools run --rm downloader-embedding`,
-  # also folded into scripts/update.sh.
-  downloader-embedding:
-    image: curlimages/curl:latest
-    profiles: ["tools"]
-    user: root
-    volumes:
-      - models:/models
-    entrypoint: ["sh", "-c"]
-    command:
-      - >
-        test -f /models/${EMBEDDING_MODEL_FILE:-nomic-embed-text-v1.5.Q8_0.gguf} &&
-        echo "already downloaded, skipping" ||
-        curl -L --fail --create-dirs -o /models/${EMBEDDING_MODEL_FILE:-nomic-embed-text-v1.5.Q8_0.gguf}
-        https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/${EMBEDDING_MODEL_FILE:-nomic-embed-text-v1.5.Q8_0.gguf}
-
   qdrant:
     image: qdrant/qdrant:latest
     container_name: qdrant
@@ -122,19 +75,20 @@ services:
     depends_on:
       qdrant:
         condition: service_healthy
-      litellm:
+      omniroute:
         condition: service_healthy
     volumes:
       - openwebui-data:/app/backend/data
     env_file: .env
     environment:
       - WEBUI_AUTH=True
-      # Routed through the litellm proxy, not llama-server directly — see issue #15.
-      # OPENAI_API_KEY must be a virtual key created for Open WebUI per
-      # docs/proxy-key-onboarding.md (name it "openwebui"), set as
-      # OPENWEBUI_LITELLM_KEY in .env.
-      - OPENAI_API_BASE_URL=http://litellm:4000/v1
-      - OPENAI_API_KEY=${OPENWEBUI_LITELLM_KEY}
+      # Routed through the omniroute gateway, not llama-server directly — see
+      # issue #15 (original rationale) and #31 (litellm -> omniroute
+      # migration). OPENAI_API_KEY must be a per-workload key created for
+      # Open WebUI in the omniroute dashboard (Keys -> Create, label
+      # "openwebui") — no scripted mint yet, see docs/proxy-key-onboarding.md.
+      - OPENAI_API_BASE_URL=http://omniroute:${OMNIROUTE_API_PORT:-20129}/v1
+      - OPENAI_API_KEY=${OPENWEBUI_OMNIROUTE_KEY}
       - VECTOR_DB=qdrant
       - QDRANT_URI=http://qdrant:6333
     ports:
@@ -142,148 +96,50 @@ services:
     restart: unless-stopped
     networks: [ai-stack]
 
-  litellm:
-    image: ghcr.io/berriai/litellm:main-stable
-    container_name: litellm
+  # Replaces litellm — see issue #31 (wayfinder map) for the full migration
+  # rationale/findings. No static config.yaml equivalent: provider routing
+  # (llama-server, searxng-search) is registered once through the dashboard
+  # or POST /api/providers after first boot, not checked into this repo —
+  # see docs/proxy-key-onboarding.md.
+  omniroute:
+    image: diegosouzapw/omniroute:latest
+    container_name: omniroute
     depends_on:
-      litellm-db:
-        condition: service_healthy
       llama-server:
         condition: service_started
-      redis:
-        condition: service_healthy
     volumes:
-      - ./litellm-config.yaml:/app/config.yaml:ro
-    # LITELLM_MASTER_KEY / LITELLM_SALT_KEY come straight from .env via env_file
-    # (names match what litellm reads). LITELLM_SALT_KEY must not change after
-    # first run — see .env.example.
+      - omniroute-data:/app/data
     env_file: .env
     environment:
-      - DATABASE_URL=postgresql://litellm:${LITELLM_DB_PASSWORD}@litellm-db:5432/litellm
-      # Setting these is all LiteLLM needs to use Redis for router state,
-      # rate limits/budgets, and cache invalidation — no extra config.yaml
-      # block required. See https://docs.litellm.ai/docs/proxy/caching.
-      - REDIS_HOST=redis
-      - REDIS_PORT=6379
-      - REDIS_PASSWORD=${REDIS_PASSWORD}
-    # The litellm container only joins the ai-stack bridge network, which has
-    # no visibility into the LAN's mDNS/local-DNS names — search.home won't
-    # resolve without this. Set SEARXNG_LAN_IP in .env to its stable LAN IP
-    # (static DHCP reservation recommended). See docs/research/litellm-searxng-search.md.
+      # Split-port mode: dashboard and API are fully separate ports (unlike
+      # LiteLLM's single :4000 for both /v1 and /ui) — only API_PORT is
+      # published below, so the dashboard has no network route in from
+      # outside this container at all. No NPM path-deny rule needed.
+      - API_HOST=0.0.0.0
+      - API_PORT=${OMNIROUTE_API_PORT:-20129}
+      - DASHBOARD_PORT=${OMNIROUTE_DASHBOARD_PORT:-20128}
+      # Required to register llama-server/searxng-search as providers —
+      # their base URLs are LAN/container-internal addresses, blocked by
+      # default (SSRF guard against public-provider spoofing).
+      - OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true
+      - OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS=true
+    # Same reasoning as litellm's extra_hosts entry below — ai-stack's bridge
+    # network can't resolve search.home on its own.
     extra_hosts:
       - "search.home:${SEARXNG_LAN_IP}"
-    command: ["--config", "/app/config.yaml", "--port", "4000"]
     ports:
-      # published for LAN access (proxy.ai.home) and, via NPM, proxy.ai.haylan.ch —
-      # NPM must deny the /ui path on the external host. See docs/network-access.md.
-      - "${LITELLM_PORT:-4000}:4000"
+      - "${OMNIROUTE_API_PORT:-20129}:${OMNIROUTE_API_PORT:-20129}"
     restart: unless-stopped
     networks: [ai-stack]
     healthcheck:
       test:
         - CMD-SHELL
-        - python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')"
+        - python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:${OMNIROUTE_API_PORT:-20129}/healthz')"
       interval: 30s
       timeout: 10s
       retries: 3
       start_period: 40s
 
-  litellm-db:
-    image: postgres:16-alpine
-    container_name: litellm-db
-    env_file: .env
-    environment:
-      - POSTGRES_USER=litellm
-      - POSTGRES_PASSWORD=${LITELLM_DB_PASSWORD}
-      - POSTGRES_DB=litellm
-    volumes:
-      - litellm-db-data:/var/lib/postgresql/data
-    restart: unless-stopped
-    networks: [ai-stack]
-    healthcheck:
-      test: ["CMD-SHELL", "pg_isready -d litellm -U litellm"]
-      interval: 5s
-      timeout: 5s
-      retries: 10
-
-  # Backs litellm's router state, rate limits/budgets, and cache
-  # invalidation (see the litellm service's REDIS_* env vars above).
-  # ponytail: no persistence volume — everything litellm stores here is
-  # cache/coordination state it's fine to lose on restart, not source data.
-  redis:
-    image: redis:7-alpine
-    container_name: redis
-    command: ["redis-server", "--requirepass", "${REDIS_PASSWORD}"]
-    restart: unless-stopped
-    networks: [ai-stack]
-    healthcheck:
-      test: ["CMD-SHELL", "redis-cli -a ${REDIS_PASSWORD} ping | grep -q PONG"]
-      interval: 5s
-      timeout: 5s
-      retries: 10
-
-  # Separate Postgres instance (with the pgvector extension) for the
-  # knowledgebase — NOT the same database as litellm-db, which is plain
-  # postgres:16-alpine and has no vector extension installed. See
-  # docs/research/litellm-knowledgebase.md.
-  pgvector-db:
-    image: pgvector/pgvector:pg16
-    container_name: pgvector-db
-    env_file: .env
-    environment:
-      - POSTGRES_USER=litellm_pgvector
-      - POSTGRES_PASSWORD=${PGVECTOR_DB_PASSWORD}
-      - POSTGRES_DB=litellm_pgvector
-    volumes:
-      - pgvector-db-data:/var/lib/postgresql/data
-    restart: unless-stopped
-    networks: [ai-stack]
-    healthcheck:
-      test: ["CMD-SHELL", "pg_isready -d litellm_pgvector -U litellm_pgvector"]
-      interval: 5s
-      timeout: 5s
-      retries: 10
-
-  # LiteLLM's native knowledgebase/vector-store feature has no Qdrant backend
-  # (the qdrant service above only serves Open WebUI's own RAG/Memory) — this
-  # companion service (github.com/BerriAI/litellm-pgvector) is the only
-  # self-hosted path. No published image exists yet, so this builds from a
-  # vendored copy in vendor/litellm-pgvector/ (see that dir's README) rather
-  # than a remote git build context — the server's Docker/BuildKit couldn't
-  # do an authenticated-looking clone of a public github.com repo (fails
-  # with "could not read Username ... terminal prompts disabled"), and
-  # vendoring sidesteps needing that debugged. See
-  # docs/research/litellm-knowledgebase.md.
-  # ponytail: unverified against real hardware — Prisma migration behavior on
-  # first boot and the exact vector_store_registry field names for the
-  # pg_vector provider need a live smoke test. See issue #24.
-  litellm-pgvector:
-    build:
-      context: ./vendor/litellm-pgvector
-    container_name: litellm-pgvector
-    depends_on:
-      pgvector-db:
-        condition: service_healthy
-      litellm:
-        condition: service_healthy
-    environment:
-      - DATABASE_URL=postgresql://litellm_pgvector:${PGVECTOR_DB_PASSWORD}@pgvector-db:5432/litellm_pgvector
-      - SERVER_API_KEY=${LITELLM_PGVECTOR_API_KEY}
-      # Calls back into litellm for embeddings, same pattern as any other
-      # workload — see docs/proxy-key-onboarding.md for issuing this key.
-      # openai/ prefix required — litellm.aembedding can't infer a provider
-      # from a bare model name plus a custom api_base (raises "LLM Provider
-      # NOT provided"), same reasoning as the openai/ prefix on
-      # qwen3.8-27b-local and local-embedding in litellm-config.yaml.
-      - EMBEDDING__MODEL=openai/local-embedding
-      - EMBEDDING__BASE_URL=http://litellm:4000
-      - EMBEDDING__API_KEY=${LITELLM_PGVECTOR_EMBEDDING_KEY}
-      - EMBEDDING__DIMENSIONS=768
-    expose:
-      - "8000"
-    restart: unless-stopped
-    networks: [ai-stack]
-
   lazytainer:
     image: ghcr.io/vmorganp/lazytainer:master
     container_name: lazytainer
@@ -311,5 +167,4 @@ volumes:
   models:
   qdrant-data:
   openwebui-data:
-  litellm-db-data:
-  pgvector-db-data:
+  omniroute-data: