feat(litellm): wire SearXNG search, pgvector knowledgebase, and memory ingestion

- search_tools block in litellm-config.yaml (SearXNG as a first-class
  search_provider, standalone /v1/search endpoint, not a model tool) plus
  extra_hosts on the litellm service so it can resolve search.home.
- New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp
  instance), pgvector-db, and litellm-pgvector services — LiteLLM's native
  knowledgebase feature has no Qdrant backend, so this is the only
  self-hosted path (docs/research/litellm-knowledgebase.md).
- vector_store_registry + local-embedding model entry in
  litellm-config.yaml, wiring it together.
- scripts/ingest-memory.sh to load data/memory.md and
  data/claude-legacy-memory.md into the knowledgebase.
- docs/memory-knowledgebase.md documenting the whole setup; data/
  gitignored (personal memory content, not meant to be committed).
- New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD,
  LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY,
  EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the
  auto-generatable ones.

Resolves #22 and #23 (wayfinder map #21). Not yet verified on real
hardware — see #24.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-02 21:38:16 +02:00
co-authored by Claude-Bot
parent a3ecbc0e02
commit f508f5670a
8 changed files with 290 additions and 5 deletions
+38
View File
@@ -22,6 +22,44 @@ model_list:
input_cost_per_token: 0.000002 # $2 / MTok
output_cost_per_token: 0.00001 # $10 / MTok
- model_name: local-embedding
litellm_params:
# Served by the dedicated embedding-server (nomic-embed-text-v1.5), not
# the chat model — see docker-compose.yml. Called by litellm-pgvector
# to embed knowledgebase content, and available directly at
# /v1/embeddings for anything else that wants it.
model: openai/local-embedding
api_base: http://embedding-server:8080/v1
api_key: local
model_info:
mode: embedding
# SearXNG-backed web search — a standalone REST endpoint (/v1/search/searxng-search),
# NOT a model-callable tool and not auto-injected into chat completions. See
# docs/research/litellm-searxng-search.md. Requires the litellm container to
# resolve search.home — see the `extra_hosts` entry in docker-compose.yml.
search_tools:
- search_tool_name: searxng-search
litellm_params:
search_provider: searxng
api_base: http://search.home/
# Knowledgebase / RAG, backed by the litellm-pgvector companion service (NOT
# Qdrant — LiteLLM's native vector-store feature has no Qdrant provider, see
# docs/research/litellm-knowledgebase.md). vector_store_id is this proxy's
# own identifier for the store, not assigned by a backend.
# ponytail: field names here (custom_llm_provider: pg_vector, api_base
# pointed at litellm-pgvector) are the best fit from the litellm-pgvector
# README, not confirmed against a running deploy yet — smoke-test before
# relying on it. See issue #24.
vector_store_registry:
- vector_store_name: memory-and-notes
litellm_params:
vector_store_id: "memory-and-notes"
custom_llm_provider: pg_vector
api_base: http://litellm-pgvector:8000
embedding_model: local-embedding
router_settings:
# ponytail: LiteLLM's request-prioritization scheduler is beta (see
# docs/proxy-request-priority.md) — exact settings key/shape must be