haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

OmniRoute: web-search tool equivalent to the SearXNG standalone endpoint #35

New Issue
Closed
opened 2026-09-03 17:18:05 +00:00 by haylan · 1 comment
haylan commented 2026-09-03 17:18:05 +00:00
Owner
Copy Link
Copy Source

Part of #31

Question

Current setup: litellm-config.yaml's search_tools block wires the LAN's SearXNG instance in as a standalone REST endpoint (/v1/search/searxng-search) — deliberately not a model-callable tool, to sidestep Qwen3.8-27B's flaky tool-calling. See docs/research/litellm-searxng-search.md.

Does OmniRoute support wiring in a self-hosted SearXNG instance the same way (a directly-callable REST endpoint, independent of whether the model can emit tool calls)? Or is web search in OmniRoute only available as a model-invoked tool — which would reopen the tool-calling reliability problem this design avoided? Does it need extra_hosts-style LAN DNS resolution the same way, or does it have its own networking model to account for?

Part of #31 ## Question Current setup: `litellm-config.yaml`'s `search_tools` block wires the LAN's SearXNG instance in as a standalone REST endpoint (`/v1/search/searxng-search`) — deliberately *not* a model-callable tool, to sidestep Qwen3.8-27B's flaky tool-calling. See `docs/research/litellm-searxng-search.md`. Does OmniRoute support wiring in a self-hosted SearXNG instance the same way (a directly-callable REST endpoint, independent of whether the model can emit tool calls)? Or is web search in OmniRoute only available as a model-invoked tool — which would reopen the tool-calling reliability problem this design avoided? Does it need `extra_hosts`-style LAN DNS resolution the same way, or does it have its own networking model to account for?
haylan added the wayfinder:research label 2026-09-03 17:18:05 +00:00
haylan self-assigned this 2026-09-03 17:20:37 +00:00
haylan commented 2026-09-03 17:21:13 +00:00
Author
Owner
Copy Link
Copy Source

Answer: Yes — OmniRoute has a native standalone REST search API equivalent to LiteLLM's search_tools, plus its own SSRF/egress opt-in for LAN targets.

1. Standalone REST endpoint, independent of model tool-calling — confirmed.

docs/reference/API_REFERENCE.md documents a dedicated Search API, a sibling of chat completions, not a chat-tool:

Method Path Description
GET /v1/search List configured search providers + capabilities
POST /v1/search Run a search query — body validated by v1SearchSchema, supports caching/coalescing
GET /v1/search/analytics Per-provider hit/latency/cache stats

Auth is a plain Bearer API key (extractApiKey / isValidApiKey / enforceApiKeyPolicy) — no chat-completion or tools=[...] involvement. Same shape as LiteLLM's /v1/search/{search_tool_name}.
Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/API_REFERENCE.md

2. SearXNG is a built-in provider, not a custom-endpoint workaround.

docs/reference/PROVIDER_REFERENCE.md lists searxng-search among 14 search providers, explicitly called out as self-hosted: "Configure your own SearXNG instance base URL... API key is optional... Some instances may require a bearer token."
Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/PROVIDER_REFERENCE.md

Confirmed directly in source, src/lib/providers/validation/searchProviders.ts: the SearXNG provider builds a plain GET request (q=...&format=json) against providerSpecificData.baseUrl (defaults to http://localhost:8888/search if unset, trailing slash normalized), with an optional Authorization: Bearer {apiKey} header if a key is configured. It is REST-only — no tool-calling grammar or model involvement anywhere in this code path.
Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/src/lib/providers/validation/searchProviders.ts

So: web search is not tool-calling-only in OmniRoute. Wiring the LAN SearXNG instance in as searxng-search with a custom baseUrl and calling POST /v1/search directly sidesteps Qwen3.8-27B's tool-calling flakiness exactly the way litellm-config.yaml's search_tools block does today — nothing requires exposing it back to the model as a function-calling tool.

3. Networking: not the same extra_hosts/mDNS story — a different, OmniRoute-specific gate.

docs/guides/DOCKER_GUIDE.md documents Docker deployment mechanics but has no mention of LAN mDNS/.local/.home hostname resolution, extra_hosts, or network_mode — i.e., it doesn't call out the same "your container can't see the LAN's local-DNS namespace" problem that docs/research/litellm-searxng-search.md had to solve for search.home. If OmniRoute itself runs in a Docker container the same underlying Docker-bridge-DNS limitation would presumably still apply and still need an extra_hosts entry (or an IP literal) — OmniRoute's docs just don't discuss it.
Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/guides/DOCKER_GUIDE.md

What OmniRoute does have that LiteLLM doesn't is its own outbound-egress/SSRF guard sitting in front of provider URLs, documented in docs/reference/FEATURE_FLAGS.md:

  • OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS (boolean, default false) — "Allow provider URLs pointing to private/internal networks."
  • OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS (boolean, default true) — "Allow adding/validating providers on local/private addresses (127.0.0.1, localhost, LAN)."
  • OUTBOUND_SSRF_GUARD_ENABLED (boolean, default true) — "Block outbound requests to private/internal IP ranges."

Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/FEATURE_FLAGS.md

So a self-hosted LAN SearXNG target needs this flag combination satisfied (LAN ranges are allowed by default; broader private ranges need OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true) on top of whatever DNS resolution is required — this is a security allowlist gate, not a DNS mechanism, and it's specific to OmniRoute; LiteLLM has no equivalent.

Bottom line

  • OmniRoute supports SearXNG the same way LiteLLM does: a native provider behind a standalone /v1/search REST endpoint, independent of model tool-calling. Switching to OmniRoute would not reopen the Qwen3.8-27B tool-calling reliability problem for search.
  • Networking is not a drop-in match: OmniRoute's docs don't address LAN mDNS/extra_hosts resolution at all (may still be needed if OmniRoute runs containerized — undocumented), and it adds its own SSRF/egress opt-in flags (OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS / OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS) that must be set correctly to reach a private/LAN SearXNG instance at all.

Confidence: Medium-high on the REST-endpoint/tool-calling-independence finding (confirmed both in docs and directly in searchProviders.ts source); medium on the networking/SSRF-flag findings (docs-only, not smoke-tested against a real deployment) — worth a live check before switching if this becomes an actual migration decision.

Sources

## Answer: Yes — OmniRoute has a native standalone REST search API equivalent to LiteLLM's `search_tools`, plus its own SSRF/egress opt-in for LAN targets. **1. Standalone REST endpoint, independent of model tool-calling — confirmed.** `docs/reference/API_REFERENCE.md` documents a dedicated Search API, a sibling of chat completions, not a chat-tool: | Method | Path | Description | |---|---|---| | GET | `/v1/search` | List configured search providers + capabilities | | POST | `/v1/search` | Run a search query — body validated by `v1SearchSchema`, supports caching/coalescing | | GET | `/v1/search/analytics` | Per-provider hit/latency/cache stats | Auth is a plain Bearer API key (`extractApiKey` / `isValidApiKey` / `enforceApiKeyPolicy`) — no chat-completion or `tools=[...]` involvement. Same shape as LiteLLM's `/v1/search/{search_tool_name}`. Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/API_REFERENCE.md **2. SearXNG is a built-in provider, not a custom-endpoint workaround.** `docs/reference/PROVIDER_REFERENCE.md` lists `searxng-search` among 14 search providers, explicitly called out as self-hosted: "Configure your own SearXNG instance base URL... API key is optional... Some instances may require a bearer token." Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/PROVIDER_REFERENCE.md Confirmed directly in source, `src/lib/providers/validation/searchProviders.ts`: the SearXNG provider builds a plain `GET` request (`q=...&format=json`) against `providerSpecificData.baseUrl` (defaults to `http://localhost:8888/search` if unset, trailing slash normalized), with an optional `Authorization: Bearer {apiKey}` header if a key is configured. It is REST-only — no tool-calling grammar or model involvement anywhere in this code path. Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/src/lib/providers/validation/searchProviders.ts **So: web search is not tool-calling-only in OmniRoute.** Wiring the LAN SearXNG instance in as `searxng-search` with a custom `baseUrl` and calling `POST /v1/search` directly sidesteps Qwen3.8-27B's tool-calling flakiness exactly the way `litellm-config.yaml`'s `search_tools` block does today — nothing requires exposing it back to the model as a function-calling tool. **3. Networking: not the same extra_hosts/mDNS story — a different, OmniRoute-specific gate.** `docs/guides/DOCKER_GUIDE.md` documents Docker deployment mechanics but has **no mention of LAN mDNS/`.local`/`.home` hostname resolution, `extra_hosts`, or `network_mode`** — i.e., it doesn't call out the same "your container can't see the LAN's local-DNS namespace" problem that `docs/research/litellm-searxng-search.md` had to solve for `search.home`. If OmniRoute itself runs in a Docker container the same underlying Docker-bridge-DNS limitation would presumably still apply and still need an `extra_hosts` entry (or an IP literal) — OmniRoute's docs just don't discuss it. Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/guides/DOCKER_GUIDE.md What OmniRoute **does** have that LiteLLM doesn't is its own outbound-egress/SSRF guard sitting in front of provider URLs, documented in `docs/reference/FEATURE_FLAGS.md`: - `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` (boolean, default `false`) — "Allow provider URLs pointing to private/internal networks." - `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS` (boolean, default `true`) — "Allow adding/validating providers on local/private addresses (127.0.0.1, localhost, LAN)." - `OUTBOUND_SSRF_GUARD_ENABLED` (boolean, default `true`) — "Block outbound requests to private/internal IP ranges." Source: https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/FEATURE_FLAGS.md So a self-hosted LAN SearXNG target needs this flag combination satisfied (LAN ranges are allowed by default; broader private ranges need `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true`) on top of whatever DNS resolution is required — this is a security allowlist gate, not a DNS mechanism, and it's specific to OmniRoute; LiteLLM has no equivalent. ## Bottom line - OmniRoute supports SearXNG the same way LiteLLM does: a native provider behind a standalone `/v1/search` REST endpoint, independent of model tool-calling. Switching to OmniRoute would **not** reopen the Qwen3.8-27B tool-calling reliability problem for search. - Networking is not a drop-in match: OmniRoute's docs don't address LAN mDNS/extra_hosts resolution at all (may still be needed if OmniRoute runs containerized — undocumented), and it adds its own SSRF/egress opt-in flags (`OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` / `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS`) that must be set correctly to reach a private/LAN SearXNG instance at all. **Confidence:** Medium-high on the REST-endpoint/tool-calling-independence finding (confirmed both in docs and directly in `searchProviders.ts` source); medium on the networking/SSRF-flag findings (docs-only, not smoke-tested against a real deployment) — worth a live check before switching if this becomes an actual migration decision. ## Sources - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/API_REFERENCE.md - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/PROVIDER_REFERENCE.md - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/ENVIRONMENT.md - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/reference/FEATURE_FLAGS.md - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/src/lib/providers/validation/searchProviders.ts - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/guides/DOCKER_GUIDE.md - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/security/GUARDRAILS.md (checked, no relevant content) - https://raw.githubusercontent.com/diegosouzapw/OmniRoute/main/docs/security/EGRESS_POLICY.md (checked, IP-family policy only, not SSRF-relevant)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
wayfinder:grilling
Wayfinder grilling ticket
 wayfinder:map
Wayfinder map issue
 wayfinder:prototype
Wayfinder prototype ticket
 wayfinder:research
Wayfinder research ticket
 wayfinder:task
Wayfinder task ticket
No labels wayfinder:research
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
haylan (Arthur Erlich)
No Assignees haylan
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: haylan/LLM-Server#35