LLM-Server/docs/research/litellm-searxng-search.md

Research: Wiring the local SearXNG instance into LiteLLM's web-search feature

Question: How does LiteLLM's web-search integration (https://docs.litellm.ai/docs/search) actually work, and what does wiring the local SearXNG instance (http://search.home/) into this repo's litellm-config.yaml require?

Answer, short version: SearXNG is a natively supported provider for LiteLLM's /v1/search feature — no custom-endpoint workaround needed. But the feature is not a model-callable tool and not automatic context-injection into chat completions either — it's a separate REST API (/v1/search/{search_tool_name}) that a caller (Open WebUI, a script, a future MCP wrapper) must invoke directly, independent of any LLM call. That sidesteps this project's known-flaky Qwen3.8-27B tool-calling entirely, as long as nothing wraps the endpoint back into model-driven tool-calling. Reachability is the real blocker: search.home is a LAN mDNS/local-DNS name that the litellm container cannot resolve by default — needs an extra_hosts entry in docker-compose.yml.

1. What LiteLLM's search feature actually is

LiteLLM ships a unified search API (/v1/search and /v1/search/{search_tool_name}) that wraps multiple search-provider backends behind one Perplexity-compatible request/response shape. Source: https://docs.litellm.ai/docs/search

Config shape in config.yaml (LiteLLM's own documented example, Perplexity shown, same shape for every provider):

search_tools:
  - search_tool_name: perplexity-search
    litellm_params:
      search_provider: perplexity
      api_key: os.environ/PERPLEXITYAI_API_KEY

Call shape:

curl http://0.0.0.0:4000/v1/search/searxng-search \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"query": "latest AI developments", "max_results": 5}'

Source: https://docs.litellm.ai/docs/search

18 providers are listed as supported, including Perplexity, Tavily, Exa AI, Brave, Parallel AI, Google PSE, DataForSEO, Firecrawl, SearXNG, Linkup, Serper, DuckDuckGo, SearchAPI.io, You.com, APISerpent, Bedrock AgentCore, Nimble, and Bing Grounding. Source: https://docs.litellm.ai/docs/search

2. Is SearXNG natively supported? — Yes

SearXNG is one of the 18 built-in search_provider values, added by BerriAI/litellm PR #16259 ("[Feat] add serxng search API provider"). Source: https://github.com/BerriAI/litellm/pull/16259

Config shape for SearXNG specifically:

search_tools:
  - search_tool_name: searxng-search
    litellm_params:
      search_provider: searxng
      api_base: https://your-searxng-instance.com

Equivalently, the base URL can be supplied via the SEARXNG_API_BASE environment variable instead of an inline api_base key — SearXNG has no API key of its own (it's an unauthenticated local meta-search engine), so this is the one provider in the list where litellm_params doesn't need a secret. Sources: https://github.com/BerriAI/litellm/pull/16259, https://docs.litellm.ai/docs/search

For this repo, the addition to litellm-config.yaml (research only — not applied here) would look like:

search_tools:
  - search_tool_name: searxng-search
    litellm_params:
      search_provider: searxng
      api_base: http://search.home/

No custom-endpoint or "generic OpenAI-compatible /v1/web_search" fallback is needed — the concern in the ticket that LiteLLM's docs "may assume a hosted provider like Tavily/Serper" turned out not to apply; SearXNG is a first-class search_provider value, same shape as every hosted one.

3. Tool-call vs. automatic injection vs. a third thing

The ticket asked to determine whether this is (a) a tool the model must explicitly call, or (b) automatic pre-retrieval/context-injection like Perplexity's own search-augmented answers. It's neither — it's a standalone REST endpoint that sits alongside /v1/chat/completions, not wired into it:

The documentation indicates this is a separate REST endpoint the application calls directly. The page presents /search as a standalone API endpoint alongside chat completions, not as an automatic injection feature. Users explicitly invoke the search endpoint; LiteLLM does not automatically inject search results into completions.

Source: https://docs.litellm.ai/docs/search (fetched content, describing the /v1/search and /v1/search/{search_tool_name} endpoints as siblings of /v1/chat/completions, not a chat-completion parameter or automatic retrieval step)

Practical effect: whatever calls this endpoint — Open WebUI's own web-search feature, a shell script, a future MCP server — does so with a plain HTTP call. LiteLLM's model routing and Qwen3.8-27B's tool-calling reliability are not in that path at all, unless something downstream chooses to expose this endpoint back to the model as a function-calling tool (e.g. an MCP wrapper that hands the model a web_search tool definition backed by this endpoint — that would reintroduce the model-must-emit-a-correct-tool-call problem, but that's a choice made one layer up, not something LiteLLM's /v1/search feature forces).

4. Network reachability: search.home from inside the litellm container

docker-compose.yml's litellm service joins only the ai-stack bridge network (networks: [ai-stack], line 116) and gets DNS resolution from Docker's embedded DNS server for that network — which resolves other containers by service/container name (llama-server, qdrant, etc., as already used at api_base: http://llama-server:8080/v1 in litellm-config.yaml line 8) but has no visibility into the LAN's mDNS/ local-DNS namespace that resolves search.home on the host machine or on LAN clients. So http://search.home/ will not resolve from inside the litellm container as configured today — this matches the ticket's suspicion, and is standard Docker bridge-networking behavior, not specific to this repo. Source: g:\_DEV\repos\LLM-Server\docker-compose.yml (litellm service, lines 94–124; networks: block, lines 154–156)

No extra_hosts, host.docker.internal, or network_mode: host pattern exists yet anywhere in this compose file to crib from — this would be the first. (One service, lazytainer, already uses network_mode: host, but for an unrelated reason — Docker-socket/host-port introspection — and switching litellm to host networking would be a much bigger blast-radius change than this ticket needs, dropping the ai-stack network isolation for every other port litellm exposes.) Source: g:\_DEV\repos\LLM-Server\docker-compose.yml lines 144–152

Recommendation: extra_hosts on the litellm service, mapping search.home to its LAN IP, e.g.:

litellm:
  ...
  extra_hosts:
    - "search.home:192.0.2.10"   # replace with SearXNG's actual LAN IP

This is the smallest, most local fix: one line, scoped to the one service that needs it, no change to network topology or isolation, and it keeps litellm-config.yaml's api_base: http://search.home/ value human-readable (matching this repo's existing preference for symbolic hostnames like ai.home / proxy.ai.home documented in docs/network-access.md) rather than hardcoding the LAN IP directly into the YAML config. The IP needs to stay in sync if SearXNG's host ever gets a new DHCP lease — same caveat that would apply to any hardcoded-IP alternative, just isolated to one extra_hosts line instead of buried in the search config.

host.docker.internal is not applicable here: that special hostname resolves to the Docker host's own IP (useful for reaching a service running directly on the host machine's loopback), not to arbitrary LAN mDNS names — it wouldn't help resolve search.home unless SearXNG happens to run on the same physical host as this compose stack.

5. Interaction with this project's known-flaky Qwen3.8-27B tool-calling

docs/research/qwen3.8-27b-tool-calling.md (2026-08-24) found, with medium- to-high confidence, that Qwen3.8-27B's tool-calling through llama.cpp inherits open/partially-fixed upstream parser bugs from the Qwen3.5 lineage (issues #21158, #20837 in ggml-org/llama.cpp) — tool calls can be emitted but not recognized, or land as inert XML inside a reasoning block, especially with thinking enabled. Source: g:\_DEV\repos\LLM-Server\docs\research\qwen3.8-27b-tool-calling.md (section 3, "Bottom line" section)

Given section 3 above (/v1/search is a standalone endpoint, not a model-tool), that flakiness has no bearing on the recommended integration path: nothing about calling POST /v1/search/searxng-search from Open WebUI or a script asks Qwen3.8-27B to emit a tool call at all. The search happens (or doesn't) independent of the model's tool-calling grammar/parser entirely.

The risk would resurface only if a different design choice is made later — e.g. wrapping this same SearXNG-backed endpoint as an MCP tool or a tools=[...] function definition handed to Qwen3.8-27B in a chat-completion request, so the model itself decides when to search. That path would inherit every bug documented in qwen3.8-27b-tool-calling.md (calls silently dropped, calls trapped inside <think> blocks, etc.) and would need the live smoke test that doc recommends before being trusted unattended. That is not what LiteLLM's /v1/search feature itself requires — it's a choice a caller could additionally make on top of it.

Recommendation

1. Add a search_tools block to litellm-config.yaml using search_provider: searxng and api_base: http://search.home/ (see section 2) — no custom/generic-endpoint workaround needed, this is a first-class supported provider.
2. Add extra_hosts: ["search.home:<LAN IP>"] to the litellm service in docker-compose.yml (see section 4) so the container can resolve the hostname; confirm the IP is stable (static DHCP reservation) since extra_hosts is a static mapping baked in at container start.
3. Treat /v1/search as a plain HTTP integration point, not a model tool — whatever calls it (Open WebUI, a script) should call the REST endpoint directly rather than exposing it to Qwen3.8-27B as a function-calling tool, to avoid inheriting this project's documented tool-calling flakiness (section 5). If model-driven search-tool-calling is wanted later, that's a separate decision that should be smoke-tested against the caveats in qwen3.8-27b-tool-calling.md first.

This is research only — litellm-config.yaml and docker-compose.yml are not modified by this doc.

Sources

• https://docs.litellm.ai/docs/search — LiteLLM search feature docs: endpoints, search_tools config shape, provider list, standalone-endpoint behavior.
• https://github.com/BerriAI/litellm/pull/16259 — SearXNG provider implementation: search_provider: searxng, api_base / SEARXNG_API_BASE config.
• g:\_DEV\repos\LLM-Server\docker-compose.yml — litellm service definition (lines 94–124), ai-stack network block (lines 154–156), lazytainer's network_mode: host precedent (lines 144–152).
• g:\_DEV\repos\LLM-Server\litellm-config.yaml — current proxy config conventions (model_list, litellm_params, api_base usage at line 8).
• g:\_DEV\repos\LLM-Server\docs\network-access.md — this repo's existing *.home / NPM hostname conventions.
• g:\_DEV\repos\LLM-Server\docs\research\qwen3.8-27b-tool-calling.md — prior findings on Qwen3.8-27B tool-calling reliability via llama.cpp.

Confidence/uncertainty summary

• High confidence: SearXNG is a native, first-class search_provider in LiteLLM's /v1/search feature (directly documented and confirmed via the implementing PR); the feature is a standalone REST endpoint separate from chat completions, not automatic context-injection and not itself a model-callable tool.
• Medium confidence: the exact field name (api_base vs. relying solely on SEARXNG_API_BASE) — two independent fetch passes against docs.litellm.ai returned slightly different renderings of the same example (one showed api_key: os.environ/SEARXNG_API_BASE, the other and the PR fetch showed api_base: <url>); the PR-sourced api_base form is treated as authoritative here since it comes from the implementing code change, but this should be smoke-tested against the actual deployed LiteLLM image (ghcr.io/berriai/litellm:main-stable) before being relied on verbatim.
• Not independently verified: SearXNG's actual LAN IP/hostname stability on this network, and whether the deployed LiteLLM version (main-stable, per docker-compose.yml line 95) already includes PR #16259 — worth a quick docker exec litellm pip show litellm / changelog check before wiring this in for real.