LLM-Server/docs/research/litellm-searxng-search.md

# Research: Wiring the local SearXNG instance into LiteLLM's web-search feature

**Question:** How does LiteLLM's web-search integration
(https://docs.litellm.ai/docs/search) actually work, and what does wiring the
local SearXNG instance (`http://search.home/`) into this repo's
`litellm-config.yaml` require?

**Answer, short version:** SearXNG **is** a natively supported provider for
LiteLLM's `/v1/search` feature — no custom-endpoint workaround needed. But the
feature is **not** a model-callable tool and **not** automatic
context-injection into chat completions either — it's a **separate REST API**
(`/v1/search/{search_tool_name}`) that a caller (Open WebUI, a script, a
future MCP wrapper) must invoke directly, independent of any LLM call. That
sidesteps this project's known-flaky Qwen3.8-27B tool-calling entirely, as
long as nothing wraps the endpoint back into model-driven tool-calling.
Reachability is the real blocker: `search.home` is a LAN mDNS/local-DNS name
that the `litellm` container cannot resolve by default — needs an
`extra_hosts` entry in `docker-compose.yml`.

## 1. What LiteLLM's search feature actually is

LiteLLM ships a **unified search API** (`/v1/search` and
`/v1/search/{search_tool_name}`) that wraps multiple search-provider backends
behind one Perplexity-compatible request/response shape.
Source: https://docs.litellm.ai/docs/search

Config shape in `config.yaml` (LiteLLM's own documented example, Perplexity
shown, same shape for every provider):

```yaml
search_tools:
  - search_tool_name: perplexity-search
    litellm_params:
      search_provider: perplexity
      api_key: os.environ/PERPLEXITYAI_API_KEY
```

Call shape:

```bash
curl http://0.0.0.0:4000/v1/search/searxng-search \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"query": "latest AI developments", "max_results": 5}'
```

Source: https://docs.litellm.ai/docs/search

**18 providers are listed as supported**, including Perplexity, Tavily, Exa
AI, Brave, Parallel AI, Google PSE, DataForSEO, Firecrawl, **SearXNG**,
Linkup, Serper, DuckDuckGo, SearchAPI.io, You.com, APISerpent, Bedrock
AgentCore, Nimble, and Bing Grounding.
Source: https://docs.litellm.ai/docs/search

## 2. Is SearXNG natively supported? — Yes

SearXNG is one of the 18 built-in `search_provider` values, added by
BerriAI/litellm PR #16259 ("[Feat] add serxng search API provider").
Source: https://github.com/BerriAI/litellm/pull/16259

Config shape for SearXNG specifically:

```yaml
search_tools:
  - search_tool_name: searxng-search
    litellm_params:
      search_provider: searxng
      api_base: https://your-searxng-instance.com
```

Equivalently, the base URL can be supplied via the `SEARXNG_API_BASE`
environment variable instead of an inline `api_base` key — SearXNG has no API
key of its own (it's an unauthenticated local meta-search engine), so this is
the one provider in the list where `litellm_params` doesn't need a secret.
Sources: https://github.com/BerriAI/litellm/pull/16259,
https://docs.litellm.ai/docs/search

**For this repo**, the addition to `litellm-config.yaml` (research only — not
applied here) would look like:

```yaml
search_tools:
  - search_tool_name: searxng-search
    litellm_params:
      search_provider: searxng
      api_base: http://search.home/
```

No custom-endpoint or "generic OpenAI-compatible /v1/web_search" fallback is
needed — the concern in the ticket that LiteLLM's docs "may assume a hosted
provider like Tavily/Serper" turned out not to apply; SearXNG is a first-class
`search_provider` value, same shape as every hosted one.

## 3. Tool-call vs. automatic injection vs. a third thing

The ticket asked to determine whether this is (a) a tool the model must
explicitly call, or (b) automatic pre-retrieval/context-injection like
Perplexity's own search-augmented answers. **It's neither** — it's a
**standalone REST endpoint** that sits alongside `/v1/chat/completions`, not
wired into it:

> The documentation indicates this is a separate REST endpoint the
> application calls directly. The page presents `/search` as a standalone API
> endpoint alongside chat completions, not as an automatic injection feature.
> Users explicitly invoke the search endpoint; LiteLLM does not automatically
> inject search results into completions.

Source: https://docs.litellm.ai/docs/search (fetched content, describing the
`/v1/search` and `/v1/search/{search_tool_name}` endpoints as siblings of
`/v1/chat/completions`, not a chat-completion parameter or automatic
retrieval step)

Practical effect: whatever calls this endpoint — Open WebUI's own web-search
feature, a shell script, a future MCP server — does so with a plain HTTP call.
**LiteLLM's model routing and Qwen3.8-27B's tool-calling reliability are not
in that path at all**, unless something downstream chooses to expose this
endpoint back to the model *as* a function-calling tool (e.g. an MCP wrapper
that hands the model a `web_search` tool definition backed by this endpoint —
that would reintroduce the model-must-emit-a-correct-tool-call problem, but
that's a choice made one layer up, not something LiteLLM's `/v1/search`
feature forces).

## 4. Network reachability: `search.home` from inside the `litellm` container

`docker-compose.yml`'s `litellm` service joins only the `ai-stack` bridge
network (`networks: [ai-stack]`, line 116) and gets DNS resolution from
Docker's embedded DNS server for that network — which resolves other
containers by service/container name (`llama-server`, `qdrant`, etc., as
already used at `api_base: http://llama-server:8080/v1` in
`litellm-config.yaml` line 8) but has **no visibility into the LAN's mDNS/
local-DNS namespace** that resolves `search.home` on the host machine or on
LAN clients. So `http://search.home/` will not resolve from inside the
`litellm` container as configured today — this matches the ticket's
suspicion, and is standard Docker bridge-networking behavior, not specific to
this repo.
Source: `g:\_DEV\repos\LLM-Server\docker-compose.yml` (litellm service, lines
94–124; `networks:` block, lines 154–156)

No `extra_hosts`, `host.docker.internal`, or `network_mode: host` pattern
exists yet anywhere in this compose file to crib from — this would be the
first. (One service, `lazytainer`, already uses `network_mode: host`, but for
an unrelated reason — Docker-socket/host-port introspection — and switching
`litellm` to host networking would be a much bigger blast-radius change than
this ticket needs, dropping the `ai-stack` network isolation for every other
port `litellm` exposes.)
Source: `g:\_DEV\repos\LLM-Server\docker-compose.yml` lines 144–152

**Recommendation: `extra_hosts` on the `litellm` service**, mapping
`search.home` to its LAN IP, e.g.:

```yaml
litellm:
  ...
  extra_hosts:
    - "search.home:192.0.2.10"   # replace with SearXNG's actual LAN IP
```

This is the smallest, most local fix: one line, scoped to the one service
that needs it, no change to network topology or isolation, and it keeps
`litellm-config.yaml`'s `api_base: http://search.home/` value human-readable
(matching this repo's existing preference for symbolic hostnames like
`ai.home` / `proxy.ai.home` documented in `docs/network-access.md`) rather
than hardcoding the LAN IP directly into the YAML config. The IP needs to stay
in sync if SearXNG's host ever gets a new DHCP lease — same caveat that would
apply to any hardcoded-IP alternative, just isolated to one `extra_hosts`
line instead of buried in the search config.

`host.docker.internal` is not applicable here: that special hostname
resolves to the Docker **host's** own IP (useful for reaching a service
running directly on the host machine's loopback), not to arbitrary LAN mDNS
names — it wouldn't help resolve `search.home` unless SearXNG happens to run
on the same physical host as this compose stack.

## 5. Interaction with this project's known-flaky Qwen3.8-27B tool-calling

`docs/research/qwen3.8-27b-tool-calling.md` (2026-08-24) found, with medium-
to-high confidence, that Qwen3.8-27B's tool-calling through llama.cpp
inherits open/partially-fixed upstream parser bugs from the Qwen3.5 lineage
(issues #21158, #20837 in `ggml-org/llama.cpp`) — tool calls can be emitted
but not recognized, or land as inert XML inside a reasoning block, especially
with thinking enabled.
Source: `g:\_DEV\repos\LLM-Server\docs\research\qwen3.8-27b-tool-calling.md`
(section 3, "Bottom line" section)

Given section 3 above (`/v1/search` is a standalone endpoint, not a
model-tool), **that flakiness has no bearing on the recommended integration
path**: nothing about calling `POST /v1/search/searxng-search` from Open
WebUI or a script asks Qwen3.8-27B to emit a tool call at all. The search
happens (or doesn't) independent of the model's tool-calling grammar/parser
entirely.

The risk **would** resurface only if a *different* design choice is made
later — e.g. wrapping this same SearXNG-backed endpoint as an MCP tool or a
`tools=[...]` function definition handed to Qwen3.8-27B in a chat-completion
request, so the model itself decides when to search. That path would inherit
every bug documented in `qwen3.8-27b-tool-calling.md` (calls silently dropped,
calls trapped inside `<think>` blocks, etc.) and would need the live smoke
test that doc recommends before being trusted unattended. **That is not what
LiteLLM's `/v1/search` feature itself requires** — it's a choice a caller
could additionally make on top of it.

## Recommendation

1. Add a `search_tools` block to `litellm-config.yaml` using
   `search_provider: searxng` and `api_base: http://search.home/` (see
   section 2) — no custom/generic-endpoint workaround needed, this is a
   first-class supported provider.
2. Add `extra_hosts: ["search.home:<LAN IP>"]` to the `litellm` service in
   `docker-compose.yml` (see section 4) so the container can resolve the
   hostname; confirm the IP is stable (static DHCP reservation) since
   `extra_hosts` is a static mapping baked in at container start.
3. Treat `/v1/search` as a plain HTTP integration point, not a model tool —
   whatever calls it (Open WebUI, a script) should call the REST endpoint
   directly rather than exposing it to Qwen3.8-27B as a function-calling
   tool, to avoid inheriting this project's documented tool-calling
   flakiness (section 5). If model-driven search-tool-calling is wanted
   later, that's a separate decision that should be smoke-tested against the
   caveats in `qwen3.8-27b-tool-calling.md` first.

This is research only — `litellm-config.yaml` and `docker-compose.yml` are
not modified by this doc.

## Sources

- https://docs.litellm.ai/docs/search — LiteLLM search feature docs:
  endpoints, `search_tools` config shape, provider list, standalone-endpoint
  behavior.
- https://github.com/BerriAI/litellm/pull/16259 — SearXNG provider
  implementation: `search_provider: searxng`, `api_base` /
  `SEARXNG_API_BASE` config.
- `g:\_DEV\repos\LLM-Server\docker-compose.yml` — `litellm` service
  definition (lines 94–124), `ai-stack` network block (lines 154–156),
  `lazytainer`'s `network_mode: host` precedent (lines 144–152).
- `g:\_DEV\repos\LLM-Server\litellm-config.yaml` — current proxy config
  conventions (`model_list`, `litellm_params`, `api_base` usage at line 8).
- `g:\_DEV\repos\LLM-Server\docs\network-access.md` — this repo's existing
  `*.home` / NPM hostname conventions.
- `g:\_DEV\repos\LLM-Server\docs\research\qwen3.8-27b-tool-calling.md` —
  prior findings on Qwen3.8-27B tool-calling reliability via llama.cpp.

## Confidence/uncertainty summary

- **High confidence:** SearXNG is a native, first-class `search_provider` in
  LiteLLM's `/v1/search` feature (directly documented and confirmed via the
  implementing PR); the feature is a standalone REST endpoint separate from
  chat completions, not automatic context-injection and not itself a
  model-callable tool.
- **Medium confidence:** the exact field name (`api_base` vs. relying solely
  on `SEARXNG_API_BASE`) — two independent fetch passes against
  docs.litellm.ai returned slightly different renderings of the same example
  (one showed `api_key: os.environ/SEARXNG_API_BASE`, the other and the PR
  fetch showed `api_base: <url>`); the PR-sourced `api_base` form is treated
  as authoritative here since it comes from the implementing code change, but
  this should be smoke-tested against the actual deployed LiteLLM image
  (`ghcr.io/berriai/litellm:main-stable`) before being relied on verbatim.
- **Not independently verified:** SearXNG's actual LAN IP/hostname stability
  on this network, and whether the deployed LiteLLM version
  (`main-stable`, per `docker-compose.yml` line 95) already includes PR
  #16259 — worth a quick `docker exec litellm pip show litellm` / changelog
  check before wiring this in for real.