feat(litellm): wire SearXNG search, pgvector knowledgebase, and memory ingestion
- search_tools block in litellm-config.yaml (SearXNG as a first-class search_provider, standalone /v1/search endpoint, not a model tool) plus extra_hosts on the litellm service so it can resolve search.home. - New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp instance), pgvector-db, and litellm-pgvector services — LiteLLM's native knowledgebase feature has no Qdrant backend, so this is the only self-hosted path (docs/research/litellm-knowledgebase.md). - vector_store_registry + local-embedding model entry in litellm-config.yaml, wiring it together. - scripts/ingest-memory.sh to load data/memory.md and data/claude-legacy-memory.md into the knowledgebase. - docs/memory-knowledgebase.md documenting the whole setup; data/ gitignored (personal memory content, not meant to be committed). - New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD, LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY, EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the auto-generatable ones. Resolves #22 and #23 (wayfinder map #21). Not yet verified on real hardware — see #24. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -22,6 +22,44 @@ model_list:
|
||||
input_cost_per_token: 0.000002 # $2 / MTok
|
||||
output_cost_per_token: 0.00001 # $10 / MTok
|
||||
|
||||
- model_name: local-embedding
|
||||
litellm_params:
|
||||
# Served by the dedicated embedding-server (nomic-embed-text-v1.5), not
|
||||
# the chat model — see docker-compose.yml. Called by litellm-pgvector
|
||||
# to embed knowledgebase content, and available directly at
|
||||
# /v1/embeddings for anything else that wants it.
|
||||
model: openai/local-embedding
|
||||
api_base: http://embedding-server:8080/v1
|
||||
api_key: local
|
||||
model_info:
|
||||
mode: embedding
|
||||
|
||||
# SearXNG-backed web search — a standalone REST endpoint (/v1/search/searxng-search),
|
||||
# NOT a model-callable tool and not auto-injected into chat completions. See
|
||||
# docs/research/litellm-searxng-search.md. Requires the litellm container to
|
||||
# resolve search.home — see the `extra_hosts` entry in docker-compose.yml.
|
||||
search_tools:
|
||||
- search_tool_name: searxng-search
|
||||
litellm_params:
|
||||
search_provider: searxng
|
||||
api_base: http://search.home/
|
||||
|
||||
# Knowledgebase / RAG, backed by the litellm-pgvector companion service (NOT
|
||||
# Qdrant — LiteLLM's native vector-store feature has no Qdrant provider, see
|
||||
# docs/research/litellm-knowledgebase.md). vector_store_id is this proxy's
|
||||
# own identifier for the store, not assigned by a backend.
|
||||
# ponytail: field names here (custom_llm_provider: pg_vector, api_base
|
||||
# pointed at litellm-pgvector) are the best fit from the litellm-pgvector
|
||||
# README, not confirmed against a running deploy yet — smoke-test before
|
||||
# relying on it. See issue #24.
|
||||
vector_store_registry:
|
||||
- vector_store_name: memory-and-notes
|
||||
litellm_params:
|
||||
vector_store_id: "memory-and-notes"
|
||||
custom_llm_provider: pg_vector
|
||||
api_base: http://litellm-pgvector:8000
|
||||
embedding_model: local-embedding
|
||||
|
||||
router_settings:
|
||||
# ponytail: LiteLLM's request-prioritization scheduler is beta (see
|
||||
# docs/proxy-request-priority.md) — exact settings key/shape must be
|
||||
|
||||
Reference in New Issue
Block a user