Answers whether SearXNG is natively supported by LiteLLM's /v1/search
feature, whether it's a model-tool or automatic retrieval, what
docker-compose.yml networking change is needed for the litellm container
to reach the LAN's search.home host, and how it interacts with this
project's known-flaky Qwen3.8-27B tool-calling.
Research only — litellm-config.yaml and docker-compose.yml are unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Answers whether the $20/mo Claude Pro subscription can be used as a
LiteLLM backend instead of a pay-per-token Anthropic API key. Finding:
no — Pro/Max OAuth credentials are contractually restricted to Claude
Code / native Anthropic apps (Consumer ToS + Usage Policy, actively
enforced server-side since March 2026). Documents the supported path:
a separate Anthropic Console API key as a model_list entry, same shape
as the existing llama.cpp entry. Same pattern applies to other
providers (ChatGPT Plus, Gemini subscriptions).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified against the live GitHub repo/docs (not secondhand): real,
active, functioning AI gateway, ~5 months old, effectively
single-maintainer, BSL 1.1. Fails two of the requirements from
docs/research/proxy-tool-choice.md / issue #9 — no Anthropic Messages
inbound endpoint (VoidLLM's own docs say Claude Code's LLM traffic
can't be routed through it) and no request-priority/queuing feature.
Recommendation: stay on LiteLLM.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Open WebUI now points at litellm instead of llama-server directly, using a
provisioned virtual key. llama-server's host port is dropped (internal-only
on the ai-stack network) since the proxy is the only intended entry point
now. docs/coding-cli-setup.md repointed at the proxy's endpoints/ports with
per-CLI virtual keys instead of the old shared dummy key.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves wayfinder ticket #8. The stack is reached over LAN/WAN hostnames,
not localhost:
- Open WebUI: ai.home (LAN) and ai.haylan.ch (external, via DMZ) routed
through the existing Nginx Proxy Manager instance to the published
WEBUI_PORT — no new reverse-proxy service needed in this repo.
- llama.cpp's raw API stays LAN-only and unregistered in NPM (no auth of
its own); coding CLIs reach it via the box's LAN IP or ai.home directly.
New docs/network-access.md documents the NPM proxy-host setup and the
LAN-only decision. coding-cli-setup.md and README updated to stop
referencing localhost.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves wayfinder ticket #6. Covers the two server endpoints (Anthropic
shim vs OpenAI-compatible), exact config for all three CLIs, and surfaces
the shared Qwen3.8-27B tool-calling risk plus OpenCode's own documented
local-backend brittleness up front rather than burying it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Answers issue #7. Confirms sst/opencode moved to anomalyco/opencode,
documents install methods, opencode.json provider config for a local
OpenAI-compatible server, and known tool-calling compatibility issues
against self-hosted backends (llama.cpp --jinja requirement, empty
tool_calls handling) plus Qwen3.8-27B-specific caveats.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the wayfinder research question: unsloth/Qwen3.8-27B-GGUF ships
UD-Q4_K_XL.gguf (17.6GB) directly. Documents VRAM footprint on 32GB
RDNA4 (R9700/gfx1201) at 32K and 128K context via llama.cpp ROCm/HIP.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
haylanandClaude-Bot