haylanandClaude-Bot
e7983f0710
This reverts commitabeadc49c8. Restores vendor/litellm-pgvector/ and the vector_store_registry wiring (in-band file_search tool-call support) at the user's request, after re-confirming against docs.litellm.ai/docs/completion/knowledgebase and litellm-pgvector's own README that pg_vector is still not an in-process vector_store_registry backend -- it requires this same standalone connector service either way, so there is no simpler 'native' path that was missed. Trading back in: 793 lines of vendored code, the untested Prisma migration, and the git-context build risk noted in VENDORED.md (all flagged as unverified against real hardware in issue #24), in exchange for the file_search in-band tool call memory-retrieval did not support. Conflicts resolved on top of later commits (Redis, update.sh key-minting fold-in): - .env.example / docs/memory-knowledgebase.md: kept the auto-mint-via- update.sh language, renamed MEMORY_RETRIEVAL_* back to LITELLM_PGVECTOR_*. - scripts/generate-secrets.sh: left deleted -- its job was folded into update.sh in24d749b, unrelated to this revert. - scripts/update.sh: renamed the MEMORY_RETRIEVAL_* secret/mint calls to LITELLM_PGVECTOR_* to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WHfjWrSEcGhCoeu6dQfDa
62 lines
2.7 KiB
Bash
62 lines
2.7 KiB
Bash
# Copy to .env and adjust — or just run ./scripts/update.sh, which creates
|
|
# .env from this file and fills in every secret/key below it can generate
|
|
# itself (see each var's comment). All values below are defaults baked into
|
|
# docker-compose.yml — only uncomment/change what you actually want to
|
|
# override.
|
|
|
|
# --- llama.cpp / model ---
|
|
LLAMA_MODEL_FILE=Qwen3.8-27B-UD-Q4_K_XL.gguf
|
|
LLAMA_GPU_LAYERS=999
|
|
# 65536 (64K) fits comfortably in 32GB VRAM alongside the model weights.
|
|
# Raise toward 131072 if you need more context; see docs/research/qwen3.8-27b-quant.md
|
|
# for the VRAM math at larger context sizes.
|
|
LLAMA_CTX_SIZE=65536
|
|
|
|
# --- Open WebUI ---
|
|
WEBUI_PORT=8008
|
|
# Minted automatically by ./scripts/update.sh — leave blank. Manual fallback:
|
|
# docs/proxy-key-onboarding.md.
|
|
OPENWEBUI_LITELLM_KEY=
|
|
|
|
# --- Lazytainer ---
|
|
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.
|
|
LAZYTAINER_INACTIVE_TIMEOUT=900
|
|
|
|
# --- Embedding model (knowledgebase, see docs/memory-knowledgebase.md) ---
|
|
EMBEDDING_MODEL_FILE=nomic-embed-text-v1.5.Q8_0.gguf
|
|
|
|
# --- SearXNG web search (see docs/research/litellm-searxng-search.md) ---
|
|
# Resolved automatically by ./scripts/update.sh from search.home on this
|
|
# host — leave blank. Only set by hand if that resolution fails (e.g.
|
|
# search.home isn't a static DHCP reservation and its IP drifted).
|
|
SEARXNG_LAN_IP=
|
|
|
|
# --- LiteLLM proxy (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
|
|
LITELLM_PORT=4000
|
|
# Random values, filled in automatically by ./scripts/update.sh — leave
|
|
# blank. LITELLM_SALT_KEY encrypts stored data; do not change it after the
|
|
# first run (existing encrypted data becomes unreadable if you do).
|
|
LITELLM_MASTER_KEY=
|
|
LITELLM_SALT_KEY=
|
|
LITELLM_DB_PASSWORD=
|
|
# Backs litellm's router state/rate-limits/budgets/cache invalidation
|
|
# (the redis service). Random value, filled in automatically — leave blank.
|
|
REDIS_PASSWORD=
|
|
# Admin UI login (https://<proxy>/ui). Without these, LiteLLM falls back to
|
|
# username "admin" / password = LITELLM_MASTER_KEY — set these instead so the
|
|
# master key never has to be typed into the browser. UI_PASSWORD is filled
|
|
# in automatically by ./scripts/update.sh if blank.
|
|
UI_USERNAME=admin
|
|
UI_PASSWORD=
|
|
|
|
# --- Knowledgebase (pgvector + litellm-pgvector, see docs/memory-knowledgebase.md) ---
|
|
# Random value, filled in automatically by ./scripts/update.sh — leave blank.
|
|
PGVECTOR_DB_PASSWORD=
|
|
# Auth key litellm-pgvector requires on its own API (its SERVER_API_KEY).
|
|
# Random value, filled in automatically by ./scripts/update.sh — leave blank.
|
|
LITELLM_PGVECTOR_API_KEY=
|
|
# A virtual key litellm-pgvector uses to call back into litellm for
|
|
# embeddings. Minted automatically by ./scripts/update.sh — leave blank.
|
|
# Manual fallback: docs/proxy-key-onboarding.md.
|
|
LITELLM_PGVECTOR_EMBEDDING_KEY=
|