Cross-checked the deployment against OmniRoute's own docs
(docs/reference/ENVIRONMENT.md, docs/guides/DOCKER_GUIDE.md) and found
three gaps from the original migration:
- OMNIROUTE_WS_BRIDGE_SECRET was entirely missing - ENVIRONMENT.md marks
it REQUIRED (production), for the internal Codex Responses WebSocket
bridge. docker compose config validated fine without it (compose
doesn't know omniroute's own required-var list), so this went
unnoticed until checking the docs directly.
- No mem_limit/OMNIROUTE_MEMORY_MB - the Docker guide is explicit that
the 1024MB default heap is dashboard-only sized; coding-agent workloads
(every client this stack has) need OMNIROUTE_MEMORY_MB=8192 and a
10+ GiB container ceiling. Set both.
- No stop_grace_period - the guide's --stop-timeout 40 equivalent, so
SQLite WAL changes checkpoint back into the main DB file on shutdown
instead of getting killed mid-write.
Redis checked and confirmed correctly absent - OmniRoute uses SQLite
only, no Redis anywhere in its docs.
Still open: whether API_PORT actually isolates /dashboard and /api/*
from the published port, or bridges everything through (see issue #31)
- OmniRoute's own ARCHITECTURE.md doesn't document split-port mode as a
real security boundary, and the live "[API Bridge] ... -> dashboard"
log line is ambiguous. Waiting on a live curl test against
proxy-ai.home before treating that as resolved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
No longer needed - every client is a coding CLI behind the OmniRoute
gateway, not a chat UI. Drops the open-webui and qdrant services,
WEBUI_PORT/OPENWEBUI_OMNIROUTE_KEY env vars, and the openwebui-data/
qdrant-data volumes. Qdrant only ever served Open WebUI's own built-in
memory/RAG (unrelated to the gateway-level knowledgebase removed in
472e3a4), so it goes too rather than sit unused.
Docs updated: README, docs/network-access.md (ai.home/ai.haylan.ch
section was entirely about Open WebUI, rewritten around the gateway),
docs/proxy-key-onboarding.md, docs/proxy-request-priority.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
LiteLLM -> OmniRoute (issue #31, wayfinder map + research tickets #32-37):
replace the litellm/litellm-db services with omniroute, split-port mode
(API_PORT published/reverse-proxied, DASHBOARD_PORT never published -
tighter than litellm's old /ui NPM path-deny rule), 5 new secrets in place
of LITELLM_MASTER_KEY/LITELLM_SALT_KEY, llama-server/searxng registered as
omniroute providers post-boot (no static config.yaml equivalent). No
scripted per-workload key minting yet - omniroute's POST /api/keys needs a
dashboard session, not a static bearer key - so OPENWEBUI_OMNIROUTE_KEY is
a manual step for now (docs/proxy-key-onboarding.md).
Caveat carried into the map and README: OmniRoute's own docs
(docs/security/STEALTH_GUIDE.md, MITM-TPROXY-DECRYPT.md, PUBLIC_CREDS.md
on its release/v3.8.51 branch) describe shipped features for AI-provider
client-detection evasion, system-wide HTTPS interception via a locally
installed root CA, and hiding credentials from secret scanners. Proceeding
anyway was an explicit, informed user decision.
Also drops the gateway-level memory/knowledgebase feature entirely (user:
"I don't need it") - litellm-pgvector, pgvector-db, embedding-server,
scripts/ingest-memory.sh, vendor/litellm-pgvector/, docs/memory-
knowledgebase.md. Open WebUI's own qdrant-backed memory/RAG is unrelated
and untouched. litellm-config.yaml deleted (was kept as a rollback
reference, but there's no rollback path to a feature being deliberately
removed).
Not yet verified against real hardware - see issue #31's open tickets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
The vector_store_registry block in litellm-config.yaml only seeds the
store into litellm's in-memory registry at boot — the Admin UI's Vector
Stores page (/ui/vector-stores) reads litellm's own DB
(LiteLLM_ManagedVectorStoresTable) instead, via /vector_store/list. A
config-only entry there gets silently deleted from memory the first time
anyone loads that page, since /vector_store/list treats the DB as the
source of truth and removes anything not also present there.
update.sh now registers it in the DB too, via POST /vector_store/new
(idempotent, same pattern as the existing virtual-key minting) — with the
literal resolved key, not the os.environ/... form used in
litellm-config.yaml, since the management API doesn't do config.yaml-style
substitution on request bodies.
Found in the process: /vector_store/update in this litellm version can't
touch litellm_params at all (VectorStoreUpdateRequest has no such field,
so PGVECTOR_API_KEY sent through it is silently dropped) — worth knowing
if this key ever needs rotating; documented in update.sh's comment.
Verified live: registered via the API, confirmed the row appears in
/vector_store/list (what the Admin UI page reads) with the real key, and
re-verified search still works end-to-end afterward.
litellm-pgvector's Dockerfile only runs 'prisma generate' (codegen) at
build time — nothing ever ran 'prisma db push' against pgvector-db, so the
vector_stores/embeddings tables plain didn't exist on a fresh deploy. Every
vector store call failed with 'relation "vector_stores" does not exist'.
update.sh now runs the push itself after bringing litellm-pgvector up.
Two bugs, either one fatal:
- litellm-pgvector's embeddings endpoints take a precomputed vector per
chunk (they don't call the embedding model themselves) — the script
posted {content, metadata} with no embedding field, guaranteed 422.
- The batch endpoint expects {"embeddings": [...]}; the script posted a
bare JSON array as the body.
Now embeds each file's chunks via LiteLLM's /v1/embeddings first (using
LITELLM_PGVECTOR_EMBEDDING_KEY, already minted for exactly this) before
batch-inserting. Also made both source files optional — a missing file is
skipped, not a hard failure, since neither exists in this checkout yet.
Today's embedding-server crash-loop (missing nomic-embed-text GGUF) was a
manual step nobody ran. update.sh now runs both downloader profiles
itself, every time, before bringing services up -- no separate command to
remember.
- docker-compose.yml: downloader/downloader-embedding commands gain a
`test -f ... && skip || curl ...` guard, so re-running update.sh never
re-downloads an existing model file.
- scripts/update.sh: runs both profiles after image pull/build, before
service recreation.
- scripts/download-model.sh removed -- folded in, redundant standalone
script.
- README.md / docs/memory-knowledgebase.md updated accordingly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
This reverts commit abeadc49c8.
Restores vendor/litellm-pgvector/ and the vector_store_registry wiring
(in-band file_search tool-call support) at the user's request, after
re-confirming against docs.litellm.ai/docs/completion/knowledgebase and
litellm-pgvector's own README that pg_vector is still not an in-process
vector_store_registry backend -- it requires this same standalone
connector service either way, so there is no simpler 'native' path that
was missed. Trading back in: 793 lines of vendored code, the untested
Prisma migration, and the git-context build risk noted in VENDORED.md
(all flagged as unverified against real hardware in issue #24), in
exchange for the file_search in-band tool call memory-retrieval did not
support.
Conflicts resolved on top of later commits (Redis, update.sh key-minting
fold-in):
- .env.example / docs/memory-knowledgebase.md: kept the auto-mint-via-
update.sh language, renamed MEMORY_RETRIEVAL_* back to
LITELLM_PGVECTOR_*.
- scripts/generate-secrets.sh: left deleted -- its job was folded into
update.sh in 24d749b, unrelated to this revert.
- scripts/update.sh: renamed the MEMORY_RETRIEVAL_* secret/mint calls to
LITELLM_PGVECTOR_* to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
New redis service (redis:7-alpine, password-protected, no persistence
volume — everything it holds is cache/coordination state). litellm gets
REDIS_HOST/REDIS_PORT/REDIS_PASSWORD, which is all LiteLLM needs to use it
for router state, rate limits/budgets, and cache invalidation — no
litellm-config.yaml block required (docs.litellm.ai/docs/proxy/caching).
REDIS_PASSWORD added to .env.example and update.sh's auto-generated
secrets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
set_if_blank/mint_key_if_blank used sed -i "s|^KEY=.*|...|", which only
replaces an existing KEY= line — a no-op if the line isn't there at all.
Any .env created before a var was added to .env.example (SEARXNG_LAN_IP,
PGVECTOR_DB_PASSWORD, MEMORY_RETRIEVAL_*, etc.) silently never got that var
filled in. Both helpers now check for "line missing" as a third case and
append it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
update.sh now creates .env from .env.example if missing, idempotently fills
in every random secret (same logic generate-secrets.sh had, now removed),
resolves SEARXNG_LAN_IP from search.home via the host's own DNS, and mints
OPENWEBUI_LITELLM_KEY / MEMORY_RETRIEVAL_EMBEDDING_KEY through LiteLLM's own
/key/generate API once litellm is up — no more manual Admin UI step for the
stack's own two workload keys. Docs updated to point at update.sh as the
one command; docs/proxy-key-onboarding.md keeps the manual/API steps as the
fallback and for onboarding other workloads.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Per docs/research/langchain-pgvector-vs-litellm-pgvector.md (issue #25):
the vendored litellm-pgvector connector (793 lines, Prisma migrations, a
fragile git-context build) is replaced by a ~90-line FastAPI service
(services/memory-retrieval/) wrapping langchain_postgres.PGVector directly
against pgvector-db. Same gateway boundary — it still calls litellm for
embeddings, nothing talks to Postgres or the model directly except this
service.
- New services/memory-retrieval/ (main.py, Dockerfile, requirements.txt):
POST /ingest, POST /query, GET /health.
- docker-compose.yml: litellm-pgvector service replaced by memory-retrieval;
pgvector-db and embedding-server untouched.
- litellm-config.yaml: vector_store_registry block removed (no
langchain_postgres provider exists to register against; callers query
memory-retrieval directly instead of an in-band file_search tool call —
that mechanism was never confirmed working per issue #24 anyway).
- scripts/ingest-memory.sh rewritten for the new /ingest endpoint (same
per-line chunking, no dedup).
- .env vars renamed: LITELLM_PGVECTOR_API_KEY/LITELLM_PGVECTOR_EMBEDDING_KEY
-> MEMORY_RETRIEVAL_API_KEY/MEMORY_RETRIEVAL_EMBEDDING_KEY.
- vendor/litellm-pgvector/ removed entirely.
- docs/memory-knowledgebase.md updated for the new setup/query flow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
One script to bring the running stack in sync after any change: git pull,
validate compose config, rebuild the litellm-pgvector local build, re-pull
images, recreate what changed. Documented in README's quickstart, and
CLAUDE.md now tells agents to run it after touching compose/config/scripts
rather than just describing a change as done.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- search_tools block in litellm-config.yaml (SearXNG as a first-class
search_provider, standalone /v1/search endpoint, not a model tool) plus
extra_hosts on the litellm service so it can resolve search.home.
- New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp
instance), pgvector-db, and litellm-pgvector services — LiteLLM's native
knowledgebase feature has no Qdrant backend, so this is the only
self-hosted path (docs/research/litellm-knowledgebase.md).
- vector_store_registry + local-embedding model entry in
litellm-config.yaml, wiring it together.
- scripts/ingest-memory.sh to load data/memory.md and
data/claude-legacy-memory.md into the knowledgebase.
- docs/memory-knowledgebase.md documenting the whole setup; data/
gitignored (personal memory content, not meant to be committed).
- New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD,
LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY,
EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the
auto-generatable ones.
Resolves#22 and #23 (wayfinder map #21). Not yet verified on real
hardware — see #24.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Without these, LiteLLM's /ui falls back to username 'admin' and the
master key as the password, forcing the master key to be typed into
a browser. generate-secrets.sh now fills UI_PASSWORD like the other
secrets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2CR8yawSf7pwAVYnjwFea
Resolves wayfinder ticket #4. Wires up the locked decisions from the map:
- llama.cpp (ghcr.io/ggml-org/llama.cpp:server-rocm, gfx1201) serving
Qwen3.8-27B-UD-Q4_K_XL.gguf, port published for direct Claude Code CLI /
Kimi CLI access alongside Open WebUI.
- Open WebUI with WEBUI_AUTH on, RAG+Memory wired to a standalone Qdrant
service.
- Lazytainer labels on llama-server for a 15 min idle-stop.
- Named Docker volumes only (models, qdrant-data, openwebui-data) — no host
bind-mounts.
- One-off 'downloader' compose profile instead of a host-side script with
its own dependencies, wrapped by scripts/download-model.sh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
haylanandClaude-Bot