Two real-hardware findings from issue #5, both researched and fixed
together (docs/research/rocm-gpu-pin-and-render-group.md):
- GPU_MAX_HW_QUEUES=1 on llama-server and llama-server-fast. Confirmed
on real hardware: either container alone is fine (3% GPU, low
power), only two concurrent HIP contexts pin the R9700 at 100%/
boost-clock (ROCm/ROCm#5706, an MES firmware bug). The env var is
validated on the exact image this stack uses, per-process by design
— applying it to both containers is the correct scope.
- group_add switched from plain names (video/render) to resolved
numeric GIDs (HOST_VIDEO_GID/HOST_RENDER_GID) on all three GPU
services. The "unable to find group render: no matching entries in
group file" error confirmed new since the second GPU service was
added is a known Docker bug (docker/cli#4714): group_add by name
resolves against the container's own /etc/group, not the host's,
and multiple GPU services starting concurrently race on that lookup.
Numeric GIDs skip resolution entirely. scripts/update.sh's existing
comfyui-only GID resolution is generalized to resolve these once for
all three services.
docker compose config -q validated (fails fast with a clear error if
the GIDs aren't resolved yet, passes once they are).
Refs #5
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MrnMEdzeQzqZE5soVEXPCx
Second, always-resident llama.cpp instance (Qwen3-4B-Instruct-2507,
Q8_0 GGUF, ~5GB VRAM) alongside the existing Qwen3.8-27B instance, for
use as qwen-code CLI's Auto Mode classifier fastModel. Model choice
researched in docs/research/fast-model-choice.md: architecturally
non-thinking (unlike Qwen3-1.7B/0.6B), --reasoning off added
defensively per a known (closed) llama.cpp misdetection bug.
- docker-compose.yml: llama-server-fast + downloader-fast services,
omniroute depends_on updated
- .env.example: LLAMA_FAST_* vars
- scripts/update.sh: runs the new downloader profile
Refs #44
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MrnMEdzeQzqZE5soVEXPCx
Closes issue #39 — with llama-server stopped and the full ~32GB R9700
available, Qwen-Image (Apache-2.0, 20B MMDiT) at FP8 precision (~25GB)
is the recommended upgrade from FLUX.1-schnell: it has the cleanest
license of the candidates and is the only one with a ComfyUI workflow
pre-validated specifically on this GPU architecture (gfx1201/R9700),
per kyuz0/amd-r9700-comfy. HunyuanImage-3.0 is ruled out (CUDA-only,
multi-GPU datacenter VRAM floor); Krea-2 flagged as promising but too
new for R9700-specific field evidence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zCwaWJQuKgXDUfRPBqDS7
Verdict: switch. See docs/research/langchain-pgvector-vs-litellm-pgvector.md
for full reasoning, LOC comparison, and a concrete replacement service spec.
Research/doc only — no compose/config/vendor changes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Answers whether SearXNG is natively supported by LiteLLM's /v1/search
feature, whether it's a model-tool or automatic retrieval, what
docker-compose.yml networking change is needed for the litellm container
to reach the LAN's search.home host, and how it interacts with this
project's known-flaky Qwen3.8-27B tool-calling.
Research only — litellm-config.yaml and docker-compose.yml are unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Answers whether the $20/mo Claude Pro subscription can be used as a
LiteLLM backend instead of a pay-per-token Anthropic API key. Finding:
no — Pro/Max OAuth credentials are contractually restricted to Claude
Code / native Anthropic apps (Consumer ToS + Usage Policy, actively
enforced server-side since March 2026). Documents the supported path:
a separate Anthropic Console API key as a model_list entry, same shape
as the existing llama.cpp entry. Same pattern applies to other
providers (ChatGPT Plus, Gemini subscriptions).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified against the live GitHub repo/docs (not secondhand): real,
active, functioning AI gateway, ~5 months old, effectively
single-maintainer, BSL 1.1. Fails two of the requirements from
docs/research/proxy-tool-choice.md / issue #9 — no Anthropic Messages
inbound endpoint (VoidLLM's own docs say Claude Code's LLM traffic
can't be routed through it) and no request-priority/queuing feature.
Recommendation: stay on LiteLLM.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Answers issue #7. Confirms sst/opencode moved to anomalyco/opencode,
documents install methods, opencode.json provider config for a local
OpenAI-compatible server, and known tool-calling compatibility issues
against self-hosted backends (llama.cpp --jinja requirement, empty
tool_calls handling) plus Qwen3.8-27B-specific caveats.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the wayfinder research question: unsloth/Qwen3.8-27B-GGUF ships
UD-Q4_K_XL.gguf (17.6GB) directly. Documents VRAM footprint on 32GB
RDNA4 (R9700/gfx1201) at 32K and 128K context via llama.cpp ROCm/HIP.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
haylanandClaude-Bot