From c01ef8965d9f0b6dd505b0ff878709e40d5b4ac9 Mon Sep 17 00:00:00 2001 From: Haylan Date: Mon, 24 Aug 2026 18:28:14 +0200 Subject: [PATCH] Fix docs: no localhost, wire up ai.home / ai.haylan.ch via existing NPM MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves wayfinder ticket #8. The stack is reached over LAN/WAN hostnames, not localhost: - Open WebUI: ai.home (LAN) and ai.haylan.ch (external, via DMZ) routed through the existing Nginx Proxy Manager instance to the published WEBUI_PORT — no new reverse-proxy service needed in this repo. - llama.cpp's raw API stays LAN-only and unregistered in NPM (no auth of its own); coding CLIs reach it via the box's LAN IP or ai.home directly. New docs/network-access.md documents the NPM proxy-host setup and the LAN-only decision. coding-cli-setup.md and README updated to stop referencing localhost. Co-Authored-By: Claude Sonnet 5 --- README.md | 6 +++--- docs/coding-cli-setup.md | 18 ++++++++++-------- docs/network-access.md | 16 ++++++++++++++++ 3 files changed, 29 insertions(+), 11 deletions(-) create mode 100644 docs/network-access.md diff --git a/README.md b/README.md index edb19ef..6daec14 100644 --- a/README.md +++ b/README.md @@ -12,9 +12,9 @@ cp .env.example .env # adjust if needed docker compose up -d ``` -- Open WebUI: http://localhost:3000 (first signup becomes the admin account — `WEBUI_AUTH` is on) -- llama.cpp OpenAI-compatible API: http://localhost:8080/v1 -- llama.cpp Anthropic Messages API (for Claude Code CLI): http://localhost:8080/v1/messages +- Open WebUI: `http://:3000` locally, or `ai.home` / `ai.haylan.ch` once routed through Nginx Proxy Manager — see [`docs/network-access.md`](docs/network-access.md). First signup becomes the admin account (`WEBUI_AUTH` is on). +- llama.cpp OpenAI-compatible API: `http://:8080/v1` — **LAN-only, not proxied**, see `docs/network-access.md`. +- llama.cpp Anthropic Messages API (for Claude Code CLI): `http://:8080/v1/messages` — same LAN-only scope. Pointing Claude Code CLI, Kimi CLI, or OpenCode CLI at the local endpoint: see [`docs/coding-cli-setup.md`](docs/coding-cli-setup.md). diff --git a/docs/coding-cli-setup.md b/docs/coding-cli-setup.md index 93e1798..101f033 100644 --- a/docs/coding-cli-setup.md +++ b/docs/coding-cli-setup.md @@ -2,11 +2,13 @@ This stack's llama.cpp server exposes two endpoints once `docker compose up` is running (see `docker-compose.yml`): -- **OpenAI-compatible**: `http://localhost:8080/v1` (or `${LLAMA_PORT}` if you changed it in `.env`) -- **Anthropic Messages API shim**: `http://localhost:8080` (adds `/v1/messages`) +- **OpenAI-compatible**: `http://:8080/v1` (or `${LLAMA_PORT}` if you changed it in `.env`) +- **Anthropic Messages API shim**: `http://:8080` (adds `/v1/messages`) Both serve the same model — `Qwen3.8-27B-UD-Q4_K_XL.gguf` — behind whichever wire format the client speaks. +`` is this machine's LAN address — its LAN IP, or `ai.home` if your local DNS resolves that hostname directly to the box. **This API is LAN-only, not reachable via `ai.haylan.ch`** — it's deliberately not registered in Nginx Proxy Manager (no auth of its own, unlike Open WebUI). See `docs/network-access.md`. If you're running a coding CLI from this machine itself, `localhost` works too. + > **Read this before relying on it for real work.** Qwen3.8-27B's tool-calling has **documented, open llama.cpp upstream bugs** (parser fails on text before ``, tool calls emitted as inert XML inside thinking blocks — see `docs/research/qwen3.8-27b-tool-calling.md`). Every setup below inherits this risk identically, regardless of which CLI or wire format you use. Don't trust it for unattended multi-step agentic work until you've run the smoke test in [issue #5](https://git.arthurerlich.de/haylan/LLM-Server/issues/5). ## Claude Code CLI @@ -14,7 +16,7 @@ Both serve the same model — `Qwen3.8-27B-UD-Q4_K_XL.gguf` — behind whichever Claude Code speaks the **Anthropic Messages API** — point it at the shim, not the OpenAI-compatible endpoint: ```bash -export ANTHROPIC_BASE_URL=http://localhost:8080 +export ANTHROPIC_BASE_URL=http://:8080 export ANTHROPIC_API_KEY=local # value is unchecked by llama.cpp, but the client requires it set claude ``` @@ -28,7 +30,7 @@ Kimi CLI speaks plain **OpenAI Chat Completions** — no shim needed. Configure ```toml [providers.openai] type = "openai" -base_url = "http://localhost:8080/v1" +base_url = "http://:8080/v1" api_key = "local" ``` @@ -53,7 +55,7 @@ curl -fsSL https://opencode.ai/install | bash "npm": "@ai-sdk/openai-compatible", "name": "llama.cpp (local)", "options": { - "baseURL": "http://localhost:8080/v1", + "baseURL": "http://:8080/v1", "apiKey": "sk-local-not-checked" }, "models": { @@ -80,8 +82,8 @@ Select the model with `llamacpp/qwen3.8-27b`. | CLI | Wire format | Endpoint | Config | |---|---|---|---| -| Claude Code | Anthropic Messages | `http://localhost:8080` | `ANTHROPIC_BASE_URL` env var | -| Kimi CLI | OpenAI Chat Completions | `http://localhost:8080/v1` | `config.toml` provider block | -| OpenCode | OpenAI Chat Completions | `http://localhost:8080/v1` | `opencode.json` provider block | +| Claude Code | Anthropic Messages | `http://:8080` | `ANTHROPIC_BASE_URL` env var | +| Kimi CLI | OpenAI Chat Completions | `http://:8080/v1` | `config.toml` provider block | +| OpenCode | OpenAI Chat Completions | `http://:8080/v1` | `opencode.json` provider block | Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/research/opencode-cli-setup.md`. diff --git a/docs/network-access.md b/docs/network-access.md new file mode 100644 index 0000000..c9f564f --- /dev/null +++ b/docs/network-access.md @@ -0,0 +1,16 @@ +# Network access: ai.home / ai.haylan.ch + +Open WebUI is meant to be reachable as **`ai.home`** on the LAN and **`ai.haylan.ch`** from outside. This stack doesn't run its own reverse proxy — it publishes Open WebUI's port to the host (`${WEBUI_PORT:-3000}`, see `docker-compose.yml`) and relies on the **existing Nginx Proxy Manager (NPM)** instance already fronting other self-hosted services on this network. + +## What to set up in NPM + +Two Proxy Hosts, both pointing at this machine's LAN IP on port `${WEBUI_PORT:-3000}` (Open WebUI): + +- **`ai.home`** — internal only, no external DNS/TLS needed unless you want it. +- **`ai.haylan.ch`** — external, reachable via the DMZ already forwarding it to NPM; let NPM issue/manage the TLS cert as it does for other services. + +## llama.cpp's raw API stays LAN-only — deliberately + +The inference API (port `${LLAMA_PORT:-8080}`) is **not** registered in NPM and is **not** reachable via `ai.haylan.ch`. It has no authentication of its own (unlike Open WebUI, which has login enabled) — putting it on the public internet would mean an unauthenticated inference endpoint. Coding-agent CLIs (Claude Code, Kimi, OpenCode — see `docs/coding-cli-setup.md`) reach it directly over the LAN, using this machine's LAN IP or `ai.home` if your local DNS resolves that hostname straight to the box (bypassing NPM, which only fronts ports 80/443). + +If you later want external CLI access too, that's a deliberate scope change — see the map ([issue #1](https://git.arthurerlich.de/haylan/LLM-Server/issues/1)) before doing it, since it changes the security posture (the raw API would need its own auth in front of it).