Cross-checked the deployment against OmniRoute's own docs
(docs/reference/ENVIRONMENT.md, docs/guides/DOCKER_GUIDE.md) and found
three gaps from the original migration:
- OMNIROUTE_WS_BRIDGE_SECRET was entirely missing - ENVIRONMENT.md marks
it REQUIRED (production), for the internal Codex Responses WebSocket
bridge. docker compose config validated fine without it (compose
doesn't know omniroute's own required-var list), so this went
unnoticed until checking the docs directly.
- No mem_limit/OMNIROUTE_MEMORY_MB - the Docker guide is explicit that
the 1024MB default heap is dashboard-only sized; coding-agent workloads
(every client this stack has) need OMNIROUTE_MEMORY_MB=8192 and a
10+ GiB container ceiling. Set both.
- No stop_grace_period - the guide's --stop-timeout 40 equivalent, so
SQLite WAL changes checkpoint back into the main DB file on shutdown
instead of getting killed mid-write.
Redis checked and confirmed correctly absent - OmniRoute uses SQLite
only, no Redis anywhere in its docs.
Still open: whether API_PORT actually isolates /dashboard and /api/*
from the published port, or bridges everything through (see issue #31)
- OmniRoute's own ARCHITECTURE.md doesn't document split-port mode as a
real security boundary, and the live "[API Bridge] ... -> dashboard"
log line is ambiguous. Waiting on a live curl test against
proxy-ai.home before treating that as resolved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
No longer needed - every client is a coding CLI behind the OmniRoute
gateway, not a chat UI. Drops the open-webui and qdrant services,
WEBUI_PORT/OPENWEBUI_OMNIROUTE_KEY env vars, and the openwebui-data/
qdrant-data volumes. Qdrant only ever served Open WebUI's own built-in
memory/RAG (unrelated to the gateway-level knowledgebase removed in
472e3a4), so it goes too rather than sit unused.
Docs updated: README, docs/network-access.md (ai.home/ai.haylan.ch
section was entirely about Open WebUI, rewritten around the gateway),
docs/proxy-key-onboarding.md, docs/proxy-request-priority.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37):
replace the litellm/litellm-db services with omniroute, split-port mode
(API_PORT published/reverse-proxied, DASHBOARD_PORT never published -
tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place
of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as
omniroute providers post-boot (no static config.yaml equivalent). No
scripted per-workload key minting yet - omniroute's POST /api/keys needs a
dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is
a manual step for now (docs/proxy-key-onboarding.md).
Caveat carried into the map and README: OmniRoute's own docs
(docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md
on its release/v3.8.51 branch) describe shipped features for AI-provider
client-detection evasion, system-wide HTTPS interception via a locally
installed root CA, and hiding credentials from secret scanners. Proceeding
anyway was an explicit, informed user decision.
Also drops the gateway-level memory/knowledgebase feature entirely (user:
"I don't need it") - litellm-pgvector, pgvector-db, embedding-server,
scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory-
knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated
and untouched. litellm-config.yaml deleted (was kept as a rollback
reference, but there's no rollback path to a feature being deliberately
removed).
Not yet verified against real hardware - see issue #31's open tickets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
The vector_store_registry block in litellm-config.yaml only seeds the
store into litellm's in-memory registry at boot — the Admin UI's Vector
Stores page (/ui/vector-stores) reads litellm's own DB
(LiteLLM_ManagedVectorStoresTable) instead, via /vector_store/list. A
config-only entry there gets silently deleted from memory the first time
anyone loads that page, since /vector_store/list treats the DB as the
source of truth and removes anything not also present there.
update.sh now registers it in the DB too, via POST /vector_store/new
(idempotent, same pattern as the existing virtual-key minting) — with the
literal resolved key, not the os.environ/... form used in
litellm-config.yaml, since the management API doesn't do config.yaml-style
substitution on request bodies.
Found in the process: /vector_store/update in this litellm version can't
touch litellm_params at all (VectorStoreUpdateRequest has no such field,
so PGVECTOR_API_KEY sent through it is silently dropped) — worth knowing
if this key ever needs rotating; documented in update.sh's comment.
Verified live: registered via the API, confirmed the row appears in
/vector_store/list (what the Admin UI page reads) with the real key, and
re-verified search still works end-to-end afterward.
litellm-pgvector's Dockerfile only runs 'prisma generate' (codegen) at
build time — nothing ever ran 'prisma db push' against pgvector-db, so the
vector_stores/embeddings tables plain didn't exist on a fresh deploy. Every
vector store call failed with 'relation "vector_stores" does not exist'.
update.sh now runs the push itself after bringing litellm-pgvector up.
Today's embedding-server crash-loop (missing nomic-embed-text GGUF) was a
manual step nobody ran. update.sh now runs both downloader profiles
itself, every time, before bringing services up -- no separate command to
remember.
- docker-compose.yml: downloader/downloader-embedding commands gain a
`test -f ... && skip || curl ...` guard, so re-running update.sh never
re-downloads an existing model file.
- scripts/update.sh: runs both profiles after image pull/build, before
service recreation.
- scripts/download-model.sh removed -- folded in, redundant standalone
script.
- README.md / docs/memory-knowledgebase.md updated accordingly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
This reverts commit abeadc49c8.
Restores vendor/litellm-pgvector/ and the vector_store_registry wiring
(in-band file_search tool-call support) at the user's request, after
re-confirming against docs.litellm.ai/docs/completion/knowledgebase and
litellm-pgvector's own README that pg_vector is still not an in-process
vector_store_registry backend -- it requires this same standalone
connector service either way, so there is no simpler 'native' path that
was missed. Trading back in: 793 lines of vendored code, the untested
Prisma migration, and the git-context build risk noted in VENDORED.md
(all flagged as unverified against real hardware in issue #24), in
exchange for the file_search in-band tool call memory-retrieval did not
support.
Conflicts resolved on top of later commits (Redis, update.sh key-minting
fold-in):
- .env.example / docs/memory-knowledgebase.md: kept the auto-mint-via-
update.sh language, renamed MEMORY_RETRIEVAL_* back to
LITELLM_PGVECTOR_*.
- scripts/generate-secrets.sh: left deleted -- its job was folded into
update.sh in 24d749b, unrelated to this revert.
- scripts/update.sh: renamed the MEMORY_RETRIEVAL_* secret/mint calls to
LITELLM_PGVECTOR_* to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
New redis service (redis:7-alpine, password-protected, no persistence
volume — everything it holds is cache/coordination state). litellm gets
REDIS_HOST/REDIS_PORT/REDIS_PASSWORD, which is all LiteLLM needs to use it
for router state, rate limits/budgets, and cache invalidation — no
litellm-config.yaml block required (docs.litellm.ai/docs/proxy/caching).
REDIS_PASSWORD added to .env.example and update.sh's auto-generated
secrets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
set_if_blank/mint_key_if_blank used sed -i "s|^KEY=.*|...|", which only
replaces an existing KEY= line — a no-op if the line isn't there at all.
Any .env created before a var was added to .env.example (SEARXNG_LAN_IP,
PGVECTOR_DB_PASSWORD, MEMORY_RETRIEVAL_*, etc.) silently never got that var
filled in. Both helpers now check for "line missing" as a third case and
append it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
update.sh now creates .env from .env.example if missing, idempotently fills
in every random secret (same logic generate-secrets.sh had, now removed),
resolves SEARXNG_LAN_IP from search.home via the host's own DNS, and mints
OPENWEBUI_LITELLM_KEY / MEMORY_RETRIEVAL_EMBEDDING_KEY through LiteLLM's own
/key/generate API once litellm is up — no more manual Admin UI step for the
stack's own two workload keys. Docs updated to point at update.sh as the
one command; docs/proxy-key-onboarding.md keeps the manual/API steps as the
fallback and for onboarding other workloads.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
One script to bring the running stack in sync after any change: git pull,
validate compose config, rebuild the litellm-pgvector local build, re-pull
images, recreate what changed. Documented in README's quickstart, and
CLAUDE.md now tells agents to run it after touching compose/config/scripts
rather than just describing a change as done.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
haylanandClaude-Bot