haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(litellm): wire SearXNG search, pgvector knowledgebase, and memory ingestion

Browse Source 
- search_tools block in litellm-config.yaml (SearXNG as a first-class
  search_provider, standalone /v1/search endpoint, not a model tool) plus
  extra_hosts on the litellm service so it can resolve search.home.
- New embedding-server (nomic-embed-text-v1.5 on a second llama.cpp
  instance), pgvector-db, and litellm-pgvector services — LiteLLM's native
  knowledgebase feature has no Qdrant backend, so this is the only
  self-hosted path (docs/research/litellm-knowledgebase.md).
- vector_store_registry + local-embedding model entry in
  litellm-config.yaml, wiring it together.
- scripts/ingest-memory.sh to load data/memory.md and
  data/claude-legacy-memory.md into the knowledgebase.
- docs/memory-knowledgebase.md documenting the whole setup; data/
  gitignored (personal memory content, not meant to be committed).
- New .env vars (SEARXNG_LAN_IP, PGVECTOR_DB_PASSWORD,
  LITELLM_PGVECTOR_API_KEY, LITELLM_PGVECTOR_EMBEDDING_KEY,
  EMBEDDING_MODEL_FILE) and generate-secrets.sh support for the
  auto-generatable ones.

Resolves #22 and #23 (wayfinder map #21). Not yet verified on real
hardware — see #24.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
haylan
2026-09-02 21:38:16 +02:00
co-authored by Claude-Bot
parent a3ecbc0e02
commit f508f5670a
8 changed files with 290 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4
View File
@@ -36,3 +36,7 @@ An [AI gateway/proxy](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) fr
- Request priority across workloads: [`docs/proxy-request-priority.md`](docs/proxy-request-priority.md).
Open WebUI and the coding CLIs (see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md)) route through the proxy now — llama-server has no published host port anymore. **Not yet verified**: none of this has been smoke-tested on real hardware (LiteLLM's priority scheduler in particular is beta — see `docs/proxy-request-priority.md`) — see [issue #17](https://git.arthurerlich.de/haylan/LLM-Server/issues/17).
### Web search, knowledgebase, and memory
The gateway also fronts SearXNG-backed web search and a pgvector-backed knowledgebase (loaded with `data/memory.md` / `data/claude-legacy-memory.md`), wired at the LiteLLM layer so every client gets them, not just Open WebUI — see [`docs/memory-knowledgebase.md`](docs/memory-knowledgebase.md). **Not yet verified on real hardware** — see [issue #24](https://git.arthurerlich.de/haylan/LLM-Server/issues/24).