haylanandClaude-Bot
b773d9cb8f
Answers issue #7. Confirms sst/opencode moved to anomalyco/opencode, documents install methods, opencode.json provider config for a local OpenAI-compatible server, and known tool-calling compatibility issues against self-hosted backends (llama.cpp --jinja requirement, empty tool_calls handling) plus Qwen3.8-27B-specific caveats. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
LLM-Server
Local AI inference stack: llama.cpp (ROCm) serving Qwen3.8-27B on an AMD Radeon AI PRO R9700, fronted by Open WebUI (RAG + Memory via Qdrant), with Lazytainer auto-suspending the inference container when idle.
See the wayfinder map (issue #1) for the full architecture rationale and open questions.
Quickstart
cp .env.example .env # adjust if needed
./scripts/download-model.sh
docker compose up -d
- Open WebUI: http://localhost:3000 (first signup becomes the admin account —
WEBUI_AUTHis on) - llama.cpp OpenAI-compatible API: http://localhost:8080/v1
- llama.cpp Anthropic Messages API (for Claude Code CLI): http://localhost:8080/v1/messages
Pointing Claude Code CLI or Kimi CLI at the local endpoint is documented separately — see issue #6 once resolved.
Known risk: Qwen3.8-27B's tool-calling reliability against llama.cpp's Anthropic shim is not yet verified (open upstream parser bugs against its model lineage) — see docs/research/qwen3.8-27b-tool-calling.md.