haylanandClaude-Bot
abeadc49c8
Per docs/research/langchain-pgvector-vs-litellm-pgvector.md (issue #25): the vendored litellm-pgvector connector (793 lines, Prisma migrations, a fragile git-context build) is replaced by a ~90-line FastAPI service (services/memory-retrieval/) wrapping langchain_postgres.PGVector directly against pgvector-db. Same gateway boundary — it still calls litellm for embeddings, nothing talks to Postgres or the model directly except this service. - New services/memory-retrieval/ (main.py, Dockerfile, requirements.txt): POST /ingest, POST /query, GET /health. - docker-compose.yml: litellm-pgvector service replaced by memory-retrieval; pgvector-db and embedding-server untouched. - litellm-config.yaml: vector_store_registry block removed (no langchain_postgres provider exists to register against; callers query memory-retrieval directly instead of an in-band file_search tool call — that mechanism was never confirmed working per issue #24 anyway). - scripts/ingest-memory.sh rewritten for the new /ingest endpoint (same per-line chunking, no dedup). - .env vars renamed: LITELLM_PGVECTOR_API_KEY/LITELLM_PGVECTOR_EMBEDDING_KEY -> MEMORY_RETRIEVAL_API_KEY/MEMORY_RETRIEVAL_EMBEDDING_KEY. - vendor/litellm-pgvector/ removed entirely. - docs/memory-knowledgebase.md updated for the new setup/query flow. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
55 lines
2.4 KiB
Bash
55 lines
2.4 KiB
Bash
# Copy to .env and adjust. All values below are defaults baked into
|
|
# docker-compose.yml — only uncomment/change what you actually want to override.
|
|
|
|
# --- llama.cpp / model ---
|
|
LLAMA_MODEL_FILE=Qwen3.8-27B-UD-Q4_K_XL.gguf
|
|
LLAMA_GPU_LAYERS=999
|
|
# 65536 (64K) fits comfortably in 32GB VRAM alongside the model weights.
|
|
# Raise toward 131072 if you need more context; see docs/research/qwen3.8-27b-quant.md
|
|
# for the VRAM math at larger context sizes.
|
|
LLAMA_CTX_SIZE=65536
|
|
|
|
# --- Open WebUI ---
|
|
WEBUI_PORT=8008
|
|
# Required — create an "openwebui" virtual key in LiteLLM's Admin UI first
|
|
# (see docs/proxy-key-onboarding.md), then paste it here.
|
|
OPENWEBUI_LITELLM_KEY=
|
|
|
|
# --- Lazytainer ---
|
|
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.
|
|
LAZYTAINER_INACTIVE_TIMEOUT=900
|
|
|
|
# --- Embedding model (knowledgebase, see docs/memory-knowledgebase.md) ---
|
|
EMBEDDING_MODEL_FILE=nomic-embed-text-v1.5.Q8_0.gguf
|
|
|
|
# --- SearXNG web search (see docs/research/litellm-searxng-search.md) ---
|
|
# Required — the litellm container can't resolve search.home's LAN mDNS
|
|
# name on its own; this becomes an extra_hosts entry. Use a static
|
|
# DHCP-reserved IP so it doesn't drift.
|
|
SEARXNG_LAN_IP=
|
|
|
|
# --- LiteLLM proxy (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
|
|
LITELLM_PORT=4000
|
|
# Required — generate real random values before first run, e.g. `openssl rand -hex 32`.
|
|
# LITELLM_SALT_KEY encrypts stored data; do not change it after the first run
|
|
# (existing encrypted data becomes unreadable if you do).
|
|
# Required — generate a real random value before first run, e.g. `openssl rand -hex 32`.
|
|
LITELLM_MASTER_KEY=
|
|
LITELLM_SALT_KEY=
|
|
LITELLM_DB_PASSWORD=
|
|
# Admin UI login (https://<proxy>/ui). Without these, LiteLLM falls back to
|
|
# username "admin" / password = LITELLM_MASTER_KEY — set these instead so the
|
|
# master key never has to be typed into the browser.
|
|
UI_USERNAME=admin
|
|
UI_PASSWORD=
|
|
|
|
# --- Knowledgebase (pgvector + memory-retrieval, see docs/memory-knowledgebase.md) ---
|
|
# Required — random values, e.g. `openssl rand -hex 32`.
|
|
PGVECTOR_DB_PASSWORD=
|
|
# Auth key memory-retrieval requires on its own API (its SERVER_API_KEY).
|
|
MEMORY_RETRIEVAL_API_KEY=
|
|
# A virtual key memory-retrieval uses to call back into litellm for
|
|
# embeddings — create it in the Admin UI like any other workload key
|
|
# (see docs/proxy-key-onboarding.md), name it "memory-retrieval".
|
|
MEMORY_RETRIEVAL_EMBEDDING_KEY=
|