feat(litellm): wire SearXNG search, pgvector knowledgebase, and memory ingestion
- search_tools block in litellm-config.yaml (SearXNG as a first-class search_provider, standalone /v1/search endpoint, not a model tool) plus extra_hosts on the litellm service so it can resolve search.home. - New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp instance), pgvector-db, and litellm-pgvector services — LiteLLM's native knowledgebase feature has no Qdrant backend, so this is the only self-hosted path (docs/research/litellm-knowledgebase.md). - vector_store_registry + local-embedding model entry in litellm-config.yaml, wiring it together. - scripts/ingest-memory.sh to load data/memory.md and data/claude-legacy-memory.md into the knowledgebase. - docs/memory-knowledgebase.md documenting the whole setup; data/ gitignored (personal memory content, not meant to be committed). - New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD, LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY, EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the auto-generatable ones. Resolves #22 and #23 (wayfinder map #21). Not yet verified on real hardware — see #24. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generates random values for the secrets docker-compose.yml requires
|
||||
# (LITELLM_MASTER_KEY, LITELLM_SALT_KEY, LITELLM_DB_PASSWORD, UI_PASSWORD)
|
||||
# and writes them into .env — creating it from .env.example first if it
|
||||
# doesn't exist.
|
||||
# (LITELLM_MASTER_KEY, LITELLM_SALT_KEY, LITELLM_DB_PASSWORD, UI_PASSWORD,
|
||||
# PGVECTOR_DB_PASSWORD, LITELLM_PGVECTOR_API_KEY) and writes them into .env —
|
||||
# creating it from .env.example first if it doesn't exist.
|
||||
#
|
||||
# ponytail: only fills in blank values, never overwrites ones you've already
|
||||
# set — safe to re-run. Re-running won't touch LITELLM_SALT_KEY once it's
|
||||
@@ -26,5 +26,7 @@ set_if_blank LITELLM_MASTER_KEY "$(openssl rand -hex 32)"
|
||||
set_if_blank LITELLM_SALT_KEY "$(openssl rand -hex 32)"
|
||||
set_if_blank LITELLM_DB_PASSWORD "$(openssl rand -hex 32)"
|
||||
set_if_blank UI_PASSWORD "$(openssl rand -hex 16)"
|
||||
set_if_blank PGVECTOR_DB_PASSWORD "$(openssl rand -hex 32)"
|
||||
set_if_blank LITELLM_PGVECTOR_API_KEY "$(openssl rand -hex 32)"
|
||||
|
||||
echo "Done. Review .env, then set OPENWEBUI_LITELLM_KEY per docs/proxy-key-onboarding.md."
|
||||
echo "Done. Review .env, then set OPENWEBUI_LITELLM_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY, and SEARXNG_LAN_IP per docs/proxy-key-onboarding.md and docs/memory-knowledgebase.md."
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env bash
|
||||
# Loads data/memory.md and data/claude-legacy-memory.md into the LiteLLM
|
||||
# knowledgebase (the "memory-and-notes" vector store, see litellm-config.yaml)
|
||||
# via litellm-pgvector's batch-embeddings endpoint.
|
||||
#
|
||||
# ponytail: one chunk per non-empty, non-heading line — both source files are
|
||||
# already one fact/paragraph per line (no hard-wrapping), so this needs no
|
||||
# real chunking logic. Re-run after editing either file; there's no dedup, so
|
||||
# this appends duplicates on a second run against unchanged content — clear
|
||||
# the store first (DELETE the vector_store_id's rows) if you need a clean
|
||||
# reload.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
[ -f .env ] && set -a && . ./.env && set +a
|
||||
|
||||
: "${LITELLM_PGVECTOR_API_KEY:?Set LITELLM_PGVECTOR_API_KEY in .env first}"
|
||||
LITELLM_PGVECTOR_URL="${LITELLM_PGVECTOR_URL:-http://localhost:8000}"
|
||||
VECTOR_STORE_ID="memory-and-notes"
|
||||
|
||||
# Must match litellm-config.yaml's vector_store_registry entry — the
|
||||
# registry just points at a store the backend must already know about.
|
||||
# Ignores failure if it already exists (no documented idempotency check).
|
||||
curl -sf -X POST "${LITELLM_PGVECTOR_URL}/v1/vector_stores" \
|
||||
-H "Authorization: Bearer ${LITELLM_PGVECTOR_API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"name\": \"${VECTOR_STORE_ID}\"}" > /dev/null 2>&1 || true
|
||||
|
||||
ingest_file() {
|
||||
local file="$1" section=""
|
||||
local batch="[]"
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
"#"*) section="${line#\# }"; section="${section#\#\# }"; continue ;;
|
||||
""|"---") continue ;;
|
||||
esac
|
||||
batch=$(jq --arg content "$line" --arg source "$file" --arg section "$section" \
|
||||
'. += [{"content": $content, "metadata": {"source": $source, "section": $section}}]' <<<"$batch")
|
||||
done < "$file"
|
||||
|
||||
echo "Ingesting $(jq 'length' <<<"$batch") chunks from $file..."
|
||||
curl -sf -X POST "${LITELLM_PGVECTOR_URL}/v1/vector_stores/${VECTOR_STORE_ID}/embeddings/batch" \
|
||||
-H "Authorization: Bearer ${LITELLM_PGVECTOR_API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$batch" > /dev/null
|
||||
}
|
||||
|
||||
ingest_file data/memory.md
|
||||
ingest_file data/claude-legacy-memory.md
|
||||
echo "Done."
|
||||
Reference in New Issue
Block a user