feat(litellm): wire SearXNG search, pgvector knowledgebase, and memory ingestion

- search_tools block in litellm-config.yaml (SearXNG as a first-class
  search_provider, standalone /v1/search endpoint, not a model tool) plus
  extra_hosts on the litellm service so it can resolve search.home.
- New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp
  instance), pgvector-db, and litellm-pgvector services — LiteLLM's native
  knowledgebase feature has no Qdrant backend, so this is the only
  self-hosted path (docs/research/litellm-knowledgebase.md).
- vector_store_registry + local-embedding model entry in
  litellm-config.yaml, wiring it together.
- scripts/ingest-memory.sh to load data/memory.md and
  data/claude-legacy-memory.md into the knowledgebase.
- docs/memory-knowledgebase.md documenting the whole setup; data/
  gitignored (personal memory content, not meant to be committed).
- New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD,
  LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY,
  EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the
  auto-generatable ones.

Resolves #22 and #23 (wayfinder map #21). Not yet verified on real
hardware — see #24.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-02 21:38:16 +02:00
co-authored by Claude-Bot
parent a3ecbc0e02
commit f508f5670a
8 changed files with 290 additions and 5 deletions
+6 -4
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
# Generates random values for the secrets docker-compose.yml requires
# (LITELLM_MASTER_KEY, LITELLM_SALT_KEY, LITELLM_DB_PASSWORD, UI_PASSWORD)
# and writes them into .env — creating it from .env.example first if it
# doesn't exist.
# (LITELLM_MASTER_KEY, LITELLM_SALT_KEY, LITELLM_DB_PASSWORD, UI_PASSWORD,
# PGVECTOR_DB_PASSWORD, LITELLM_PGVECTOR_API_KEY) and writes them into .env —
# creating it from .env.example first if it doesn't exist.
#
# ponytail: only fills in blank values, never overwrites ones you've already
# set — safe to re-run. Re-running won't touch LITELLM_SALT_KEY once it's
@@ -26,5 +26,7 @@ set_if_blank LITELLM_MASTER_KEY "$(openssl rand -hex 32)"
set_if_blank LITELLM_SALT_KEY "$(openssl rand -hex 32)"
set_if_blank LITELLM_DB_PASSWORD "$(openssl rand -hex 32)"
set_if_blank UI_PASSWORD "$(openssl rand -hex 16)"
set_if_blank PGVECTOR_DB_PASSWORD "$(openssl rand -hex 32)"
set_if_blank LITELLM_PGVECTOR_API_KEY "$(openssl rand -hex 32)"
echo "Done. Review .env, then set OPENWEBUI_LITELLM_KEY per docs/proxy-key-onboarding.md."
echo "Done. Review .env, then set OPENWEBUI_LITELLM_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY, and SEARXNG_LAN_IP per docs/proxy-key-onboarding.md and docs/memory-knowledgebase.md."
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Loads data/memory.md and data/claude-legacy-memory.md into the LiteLLM
# knowledgebase (the "memory-and-notes" vector store, see litellm-config.yaml)
# via litellm-pgvector's batch-embeddings endpoint.
#
# ponytail: one chunk per non-empty, non-heading line — both source files are
# already one fact/paragraph per line (no hard-wrapping), so this needs no
# real chunking logic. Re-run after editing either file; there's no dedup, so
# this appends duplicates on a second run against unchanged content — clear
# the store first (DELETE the vector_store_id's rows) if you need a clean
# reload.
set -euo pipefail
cd "$(dirname "$0")/.."
[ -f .env ] && set -a && . ./.env && set +a
: "${LITELLM_PGVECTOR_API_KEY:?Set LITELLM_PGVECTOR_API_KEY in .env first}"
LITELLM_PGVECTOR_URL="${LITELLM_PGVECTOR_URL:-http://localhost:8000}"
VECTOR_STORE_ID="memory-and-notes"
# Must match litellm-config.yaml's vector_store_registry entry — the
# registry just points at a store the backend must already know about.
# Ignores failure if it already exists (no documented idempotency check).
curl -sf -X POST "${LITELLM_PGVECTOR_URL}/v1/vector_stores" \
-H "Authorization: Bearer ${LITELLM_PGVECTOR_API_KEY}" \
-H "Content-Type: application/json" \
-d "{\"name\": \"${VECTOR_STORE_ID}\"}" > /dev/null 2>&1 || true
ingest_file() {
local file="$1" section=""
local batch="[]"
while IFS= read -r line; do
case "$line" in
"#"*) section="${line#\# }"; section="${section#\#\# }"; continue ;;
""|"---") continue ;;
esac
batch=$(jq --arg content "$line" --arg source "$file" --arg section "$section" \
'. += [{"content": $content, "metadata": {"source": $source, "section": $section}}]' <<<"$batch")
done < "$file"
echo "Ingesting $(jq 'length' <<<"$batch") chunks from $file..."
curl -sf -X POST "${LITELLM_PGVECTOR_URL}/v1/vector_stores/${VECTOR_STORE_ID}/embeddings/batch" \
-H "Authorization: Bearer ${LITELLM_PGVECTOR_API_KEY}" \
-H "Content-Type: application/json" \
-d "$batch" > /dev/null
}
ingest_file data/memory.md
ingest_file data/claude-legacy-memory.md
echo "Done."