Author docker-compose stack: llama.cpp (ROCm) + Open WebUI + Qdrant + Lazytainer
Resolves wayfinder ticket #4. Wires up the locked decisions from the map: - llama.cpp (ghcr.io/ggml-org/llama.cpp:server-rocm, gfx1201) serving Qwen3.8-27B-UD-Q4_K_XL.gguf, port published for direct Claude Code CLI / Kimi CLI access alongside Open WebUI. - Open WebUI with WEBUI_AUTH on, RAG+Memory wired to a standalone Qdrant service. - Lazytainer labels on llama-server for a 15 min idle-stop. - Named Docker volumes only (models, qdrant-data, openwebui-data) — no host bind-mounts. - One-off 'downloader' compose profile instead of a host-side script with its own dependencies, wrapped by scripts/download-model.sh. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
# Copy to .env and adjust. All values below are defaults baked into
|
||||
# docker-compose.yml — only uncomment/change what you actually want to override.
|
||||
|
||||
# --- llama.cpp / model ---
|
||||
LLAMA_MODEL_FILE=Qwen3.8-27B-UD-Q4_K_XL.gguf
|
||||
LLAMA_GPU_LAYERS=999
|
||||
# 65536 (64K) fits comfortably in 32GB VRAM alongside the model weights.
|
||||
# Raise toward 131072 if you need more context; see docs/research/qwen3.8-27b-quant.md
|
||||
# for the VRAM math at larger context sizes.
|
||||
LLAMA_CTX_SIZE=65536
|
||||
LLAMA_PORT=8080
|
||||
|
||||
# --- Open WebUI ---
|
||||
WEBUI_PORT=3000
|
||||
# Dummy key — llama.cpp's OpenAI-compatible endpoint doesn't check it, but
|
||||
# Open WebUI requires the field to be non-empty.
|
||||
OPENAI_API_KEY=local
|
||||
|
||||
# --- Lazytainer ---
|
||||
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.
|
||||
LAZYTAINER_INACTIVE_TIMEOUT=900
|
||||
Reference in New Issue
Block a user