How does LiteLLM's SearXNG web-search integration work, and what does wiring it in require? #22

New Issue

2026-09-02T19:18:54Z

haylan commented

2026-09-02 19:18:54 +00:00

Part of #21

Question

How does LiteLLM's web search integration (https://docs.litellm.ai/docs/search) actually work, and what does wiring the local SearXNG instance (http://search.home/) into this proxy's litellm-config.yaml require — config shape, whether it's exposed as a tool the model must call vs. an automatic pre-retrieval step, network reachability from the litellm container to search.home on the LAN (not the ai-stack docker network), and any implications for Qwen3.8-27B's already-known-flaky tool-calling (see docs/research/qwen3.8-27b-tool-calling.md).

Part of #21 ## Question How does LiteLLM's web search integration (https://docs.litellm.ai/docs/search) actually work, and what does wiring the local SearXNG instance (`http://search.home/`) into this proxy's `litellm-config.yaml` require — config shape, whether it's exposed as a tool the model must call vs. an automatic pre-retrieval step, network reachability from the `litellm` container to `search.home` on the LAN (not the `ai-stack` docker network), and any implications for Qwen3.8-27B's already-known-flaky tool-calling (see `docs/research/qwen3.8-27b-tool-calling.md`).

haylan added the wayfinder:research label 2026-09-02 19:18:54 +00:00

haylan referenced this issue

2026-09-02 19:22:13 +00:00

LiteLLM gateway: search, knowledgebase, and gateway-level memory #21

haylan commented

2026-09-02 19:26:32 +00:00

Findings (full doc: https://git.arthurerlich.de/haylan/LLM-Server/src/branch/research/litellm-searxng-search/docs/research/litellm-searxng-search.md)

SearXNG is natively supported. LiteLLM's /v1/search feature has SearXNG as a first-class search_provider (PR BerriAI/litellm#16259) — no custom-endpoint workaround needed. Config shape:
```
search_tools:
  - search_tool_name: searxng-search
    litellm_params:
      search_provider: searxng
      api_base: http://search.home/
```
It's neither tool-call nor auto-injection — it's a third thing. /v1/search is a standalone REST endpoint alongside /v1/chat/completions, not wired into chat completions and not exposed to the model as a function-calling tool. Whatever calls it (Open WebUI, a script) does a plain HTTP call — LiteLLM does not inject results into completions automatically, and the model is never asked to emit a tool call to trigger it.
Network reachability needs a fix. search.home is a LAN-only hostname; the litellm container (on the ai-stack bridge network, using Docker's embedded DNS) can't resolve it as-is. Recommend adding extra_hosts: ["search.home:<LAN IP>"] to the litellm service in docker-compose.yml — smallest fix, keeps the symbolic hostname in config, no topology change. host.docker.internal doesn't apply (resolves to the Docker host, not arbitrary LAN mDNS names).
Qwen3.8-27B's flaky tool-calling is a non-issue for this path. Since /v1/search isn't a model tool, none of the documented llama.cpp/Qwen3.5-lineage tool-call parser bugs (docs/research/qwen3.8-27b-tool-calling.md) come into play. The risk only resurfaces if someone later wraps this endpoint as an MCP/function-calling tool handed to the model — that would be a separate decision needing its own smoke test.

Research only, no config files changed.

**Findings (full doc: https://git.arthurerlich.de/haylan/LLM-Server/src/branch/research/litellm-searxng-search/docs/research/litellm-searxng-search.md)** - **SearXNG is natively supported.** LiteLLM's `/v1/search` feature has SearXNG as a first-class `search_provider` (PR BerriAI/litellm#16259) — no custom-endpoint workaround needed. Config shape: ```yaml search_tools: - search_tool_name: searxng-search litellm_params: search_provider: searxng api_base: http://search.home/ ``` - **It's neither tool-call nor auto-injection — it's a third thing.** `/v1/search` is a standalone REST endpoint alongside `/v1/chat/completions`, not wired into chat completions and not exposed to the model as a function-calling tool. Whatever calls it (Open WebUI, a script) does a plain HTTP call — LiteLLM does not inject results into completions automatically, and the model is never asked to emit a tool call to trigger it. - **Network reachability needs a fix.** `search.home` is a LAN-only hostname; the `litellm` container (on the `ai-stack` bridge network, using Docker's embedded DNS) can't resolve it as-is. Recommend adding `extra_hosts: ["search.home:<LAN IP>"]` to the `litellm` service in docker-compose.yml — smallest fix, keeps the symbolic hostname in config, no topology change. `host.docker.internal` doesn't apply (resolves to the Docker host, not arbitrary LAN mDNS names). - **Qwen3.8-27B's flaky tool-calling is a non-issue for this path.** Since `/v1/search` isn't a model tool, none of the documented llama.cpp/Qwen3.5-lineage tool-call parser bugs (docs/research/qwen3.8-27b-tool-calling.md) come into play. The risk only resurfaces if someone later wraps this endpoint as an MCP/function-calling tool handed to the model — that would be a separate decision needing its own smoke test. Research only, no config files changed.

haylan referenced this issue from a commit

2026-09-02 19:27:24 +00:00

research: LiteLLM web-search + SearXNG wiring (issue #22)

haylan referenced this issue from a commit

2026-09-02 19:38:40 +00:00

feat(litellm): wire SearXNG search, pgvector knowledgebase, and memory ingestion

haylan closed this issue

2026-09-02 19:38:40 +00:00

Sign in to join this conversation.