Author docker-compose stack: llama.cpp (ROCm) + Open WebUI + Qdrant + Lazytainer

Resolves wayfinder ticket #4. Wires up the locked decisions from the map:
- llama.cpp (ghcr.io/ggml-org/llama.cpp:server-rocm, gfx1201) serving
  Qwen3.8-27B-UD-Q4_K_XL.gguf, port published for direct Claude Code CLI /
  Kimi CLI access alongside Open WebUI.
- Open WebUI with WEBUI_AUTH on, RAG+Memory wired to a standalone Qdrant
  service.
- Lazytainer labels on llama-server for a 15 min idle-stop.
- Named Docker volumes only (models, qdrant-data, openwebui-data) — no host
  bind-mounts.
- One-off 'downloader' compose profile instead of a host-side script with
  its own dependencies, wrapped by scripts/download-model.sh.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-24 13:01:00 +02:00
co-authored by Claude-Bot
parent 19d62aefd3
commit f0e473048d
4 changed files with 147 additions and 0 deletions
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Downloads the model GGUF straight into the `models` named volume via a
# one-off container — no huggingface-cli or host bind-mount needed.
#
# ponytail: hardcodes the one model this stack is built for (see the
# `downloader` service in docker-compose.yml for the actual URL/filename).
# Set LLAMA_MODEL_FILE in .env first if you're using a different quant.
set -euo pipefail
cd "$(dirname "$0")/.."
docker compose --profile tools run --rm downloader
echo "Model downloaded into the 'models' volume."