LLM-Server

Local AI inference stack: llama.cpp (ROCm) serving Qwen3.8-27B on an AMD Radeon AI PRO R9700, fronted by Open WebUI (RAG + Memory via Qdrant), with Lazytainer auto-suspending the inference container when idle.

See the wayfinder map (issue #1) for the full architecture rationale and open questions.

Quickstart

cp .env.example .env      # adjust if needed
./scripts/download-model.sh
docker compose up -d

• Open WebUI: http://<this-machine>:3000 locally, or ai.home / ai.haylan.ch once routed through Nginx Proxy Manager — see docs/network-access.md. First signup becomes the admin account (WEBUI_AUTH is on).
• llama.cpp OpenAI-compatible API: http://<this-machine>:8080/v1 — LAN-only, not proxied, see docs/network-access.md.
• llama.cpp Anthropic Messages API (for Claude Code CLI): http://<this-machine>:8080/v1/messages — same LAN-only scope.

Pointing Claude Code CLI, Kimi CLI, or OpenCode CLI at the local endpoint: see docs/coding-cli-setup.md.

Known risk: Qwen3.8-27B's tool-calling reliability against llama.cpp's Anthropic shim is not yet verified (open upstream parser bugs against its model lineage) — see docs/research/qwen3.8-27b-tool-calling.md.