New redis service (redis:7-alpine, password-protected, no persistence
volume — everything it holds is cache/coordination state). litellm gets
REDIS_HOST/REDIS_PORT/REDIS_PASSWORD, which is all LiteLLM needs to use it
for router state, rate limits/budgets, and cache invalidation — no
litellm-config.yaml block required (docs.litellm.ai/docs/proxy/caching).
REDIS_PASSWORD added to .env.example and update.sh's auto-generated
secrets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
update.sh now creates .env from .env.example if missing, idempotently fills
in every random secret (same logic generate-secrets.sh had, now removed),
resolves SEARXNG_LAN_IP from search.home via the host's own DNS, and mints
OPENWEBUI_LITELLM_KEY / MEMORY_RETRIEVAL_EMBEDDING_KEY through LiteLLM's own
/key/generate API once litellm is up — no more manual Admin UI step for the
stack's own two workload keys. Docs updated to point at update.sh as the
one command; docs/proxy-key-onboarding.md keeps the manual/API steps as the
fallback and for onboarding other workloads.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Per docs/research/langchain-pgvector-vs-litellm-pgvector.md (issue #25):
the vendored litellm-pgvector connector (793 lines, Prisma migrations, a
fragile git-context build) is replaced by a ~90-line FastAPI service
(services/memory-retrieval/) wrapping langchain_postgres.PGVector directly
against pgvector-db. Same gateway boundary — it still calls litellm for
embeddings, nothing talks to Postgres or the model directly except this
service.
- New services/memory-retrieval/ (main.py, Dockerfile, requirements.txt):
POST /ingest, POST /query, GET /health.
- docker-compose.yml: litellm-pgvector service replaced by memory-retrieval;
pgvector-db and embedding-server untouched.
- litellm-config.yaml: vector_store_registry block removed (no
langchain_postgres provider exists to register against; callers query
memory-retrieval directly instead of an in-band file_search tool call —
that mechanism was never confirmed working per issue #24 anyway).
- scripts/ingest-memory.sh rewritten for the new /ingest endpoint (same
per-line chunking, no dedup).
- .env vars renamed: LITELLM_PGVECTOR_API_KEY/LITELLM_PGVECTOR_EMBEDDING_KEY
-> MEMORY_RETRIEVAL_API_KEY/MEMORY_RETRIEVAL_EMBEDDING_KEY.
- vendor/litellm-pgvector/ removed entirely.
- docs/memory-knowledgebase.md updated for the new setup/query flow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- search_tools block in litellm-config.yaml (SearXNG as a first-class
search_provider, standalone /v1/search endpoint, not a model tool) plus
extra_hosts on the litellm service so it can resolve search.home.
- New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp
instance), pgvector-db, and litellm-pgvector services — LiteLLM's native
knowledgebase feature has no Qdrant backend, so this is the only
self-hosted path (docs/research/litellm-knowledgebase.md).
- vector_store_registry + local-embedding model entry in
litellm-config.yaml, wiring it together.
- scripts/ingest-memory.sh to load data/memory.md and
data/claude-legacy-memory.md into the knowledgebase.
- docs/memory-knowledgebase.md documenting the whole setup; data/
gitignored (personal memory content, not meant to be committed).
- New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD,
LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY,
EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the
auto-generatable ones.
Resolves#22 and #23 (wayfinder map #21). Not yet verified on real
hardware — see #24.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Without these, LiteLLM's /ui falls back to username 'admin' and the
master key as the password, forcing the master key to be typed into
a browser. generate-secrets.sh now fills UI_PASSWORD like the other
secrets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2CR8yawSf7pwAVYnjwFea
Open WebUI now points at litellm instead of llama-server directly, using a
provisioned virtual key. llama-server's host port is dropped (internal-only
on the ai-stack network) since the proxy is the only intended entry point
now. docs/coding-cli-setup.md repointed at the proxy's endpoints/ports with
per-CLI virtual keys instead of the old shared dummy key.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds litellm + litellm-db to docker-compose.yml, litellm-config.yaml with
custom shadow-cost pricing (Claude Sonnet 5 reference, per #11) and a
priority-scheduling stub (per #16, needs real-hardware smoke test), and
required LITELLM_MASTER_KEY/SALT_KEY/DB_PASSWORD env vars. Untested on real
hardware — that's #17. Open WebUI/coding CLIs still talk to llama.cpp
directly, migration is #15.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves wayfinder ticket #4. Wires up the locked decisions from the map:
- llama.cpp (ghcr.io/ggml-org/llama.cpp:server-rocm, gfx1201) serving
Qwen3.8-27B-UD-Q4_K_XL.gguf, port published for direct Claude Code CLI /
Kimi CLI access alongside Open WebUI.
- Open WebUI with WEBUI_AUTH on, RAG+Memory wired to a standalone Qdrant
service.
- Lazytainer labels on llama-server for a 15 min idle-stop.
- Named Docker volumes only (models, qdrant-data, openwebui-data) — no host
bind-mounts.
- One-off 'downloader' compose profile instead of a host-side script with
its own dependencies, wrapped by scripts/download-model.sh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
haylanandClaude-Bot