fix(omniroute): keep the gateway published on host port 4000
Not omniroute's own internal port (API_PORT stays at its default 20129, unreconfigured) - just the Docker port mapping, so existing NPM/firewall config pointed at :4000 keeps working without changes on that end. New OMNIROUTE_PORT env var is the host side of "OMNIROUTE_PORT:API_PORT" in docker-compose.yml's ports: entry. Also corrected docs/proxy-key-onboarding.md's dashboard-access instructions - DASHBOARD_PORT was never published to the host in the first place, so "http://<host>:20128" was never actually reachable as written; documented reaching it via the container's own bridge-network IP or an SSH port-forward instead. llama-server remains unexposed (no ports: entry, only expose:) - unaffected by this change, confirming it stays that way. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPZ6TogJiYxG8E4EQBB197
This commit is contained in:
+7
-3
@@ -39,9 +39,13 @@ LAZYTAINER_INACTIVE_TIMEOUT=900
|
||||
SEARXNG_LAN_IP=
|
||||
|
||||
# --- OmniRoute gateway (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
|
||||
# API_PORT is the only port published to the host/internet (reverse-proxied
|
||||
# by NPM) — the dashboard (DASHBOARD_PORT) is never published, see
|
||||
# docker-compose.yml's omniroute service comment.
|
||||
# OMNIROUTE_PORT is the host-published port (reverse-proxied by NPM) — kept
|
||||
# at 4000, same as the old LiteLLM setup, so existing NPM/firewall config
|
||||
# doesn't need to change. It's mapped via plain Docker port publishing onto
|
||||
# API_PORT, omniroute's own container-internal port (left at its default,
|
||||
# not reconfigured to match). The dashboard (DASHBOARD_PORT) is never
|
||||
# published at all — see docker-compose.yml's omniroute service comment.
|
||||
OMNIROUTE_PORT=4000
|
||||
OMNIROUTE_API_PORT=20129
|
||||
OMNIROUTE_DASHBOARD_PORT=20128
|
||||
# Random values, filled in automatically by ./scripts/update.sh — leave
|
||||
|
||||
@@ -22,7 +22,7 @@ Pointing Claude Code CLI, Kimi CLI, or OpenCode CLI at the local endpoint: see [
|
||||
|
||||
An [AI gateway/proxy](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) fronts llama.cpp: per-workload API keys and usage tracking. As of [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) this is [OmniRoute](https://github.com/diegosouzapw/OmniRoute), replacing the original LiteLLM setup. `./scripts/update.sh` handles most of OmniRoute's secrets (see `.env.example`); per-workload API keys still need minting by hand in the dashboard — see [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md).
|
||||
|
||||
- Gateway API: `http://<this-machine>:${OMNIROUTE_API_PORT:-20129}/v1` locally, or `proxy.ai.home` / `proxy.ai.haylan.ch` once routed through NPM — see [`docs/network-access.md`](docs/network-access.md).
|
||||
- Gateway API: `http://<this-machine>:${OMNIROUTE_PORT:-4000}/v1` locally, or `proxy.ai.home` / `proxy.ai.haylan.ch` once routed through NPM — see [`docs/network-access.md`](docs/network-access.md).
|
||||
- Dashboard (key/provider management): LAN/host-only, never published to the internet — see `docs/network-access.md`.
|
||||
- Issuing a key for a new workload: [`docs/proxy-key-onboarding.md`](docs/proxy-key-onboarding.md).
|
||||
|
||||
|
||||
+10
-4
@@ -72,9 +72,10 @@ services:
|
||||
env_file: .env
|
||||
environment:
|
||||
# Split-port mode: dashboard and API are fully separate ports (unlike
|
||||
# LiteLLM's single :4000 for both /v1 and /ui) — only API_PORT is
|
||||
# published below, so the dashboard has no network route in from
|
||||
# outside this container at all. No NPM path-deny rule needed.
|
||||
# LiteLLM's single :4000 for both /v1 and /ui) — only API_PORT (via
|
||||
# the OMNIROUTE_PORT:4000 host mapping below) is published, so the
|
||||
# dashboard has no network route in from outside this container at
|
||||
# all. No NPM path-deny rule needed.
|
||||
- API_HOST=0.0.0.0
|
||||
- API_PORT=${OMNIROUTE_API_PORT:-20129}
|
||||
- DASHBOARD_PORT=${OMNIROUTE_DASHBOARD_PORT:-20128}
|
||||
@@ -87,8 +88,13 @@ services:
|
||||
# network can't resolve search.home on its own.
|
||||
extra_hosts:
|
||||
- "search.home:${SEARXNG_LAN_IP}"
|
||||
# OMNIROUTE_PORT (host) is decoupled from API_PORT (container-internal,
|
||||
# left at omniroute's own default) via plain Docker port mapping — the
|
||||
# gateway needs to stay reachable at the same :4000 this stack has
|
||||
# always published (NPM/firewall already point there), without having
|
||||
# to also change omniroute's own internal port config to match.
|
||||
ports:
|
||||
- "${OMNIROUTE_API_PORT:-20129}:${OMNIROUTE_API_PORT:-20129}"
|
||||
- "${OMNIROUTE_PORT:-4000}:${OMNIROUTE_API_PORT:-20129}"
|
||||
restart: unless-stopped
|
||||
networks: [ai-stack]
|
||||
# ponytail: TCP-connect check, not an HTTP /healthz GET — the image has
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
This stack routes through the [AI gateway](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) (OmniRoute, see issue #31) rather than talking to llama.cpp directly — llama.cpp's own port is internal-only now (see `docker-compose.yml`). The gateway exposes:
|
||||
|
||||
- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_API_PORT:-20129}/v1`
|
||||
- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_API_PORT:-20129}`
|
||||
- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1`
|
||||
- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_PORT:-4000}`
|
||||
|
||||
Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, registered in the gateway (naming is yours to pick when adding the llama-cpp provider connection — this doc assumes `qwen3.8-27b-local` for continuity) — behind whichever wire format the client speaks.
|
||||
|
||||
@@ -18,7 +18,7 @@ Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, register
|
||||
Claude Code speaks the **Anthropic Messages API** — point it at the gateway's unified endpoint, not llama.cpp directly:
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_BASE_URL=http://<ai-box>:${OMNIROUTE_API_PORT:-20129}
|
||||
export ANTHROPIC_BASE_URL=http://<ai-box>:${OMNIROUTE_PORT:-4000}
|
||||
export ANTHROPIC_API_KEY=<claude-code-cli virtual key>
|
||||
claude
|
||||
```
|
||||
@@ -32,7 +32,7 @@ Kimi CLI speaks plain **OpenAI Chat Completions**. Configure a provider block in
|
||||
```toml
|
||||
[providers.openai]
|
||||
type = "openai"
|
||||
base_url = "http://<ai-box>:${OMNIROUTE_API_PORT:-20129}/v1"
|
||||
base_url = "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
|
||||
api_key = "<kimi-cli virtual key>"
|
||||
```
|
||||
|
||||
@@ -57,7 +57,7 @@ curl -fsSL https://opencode.ai/install | bash
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "AI proxy (local)",
|
||||
"options": {
|
||||
"baseURL": "http://<ai-box>:${OMNIROUTE_API_PORT:-20129}/v1",
|
||||
"baseURL": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
|
||||
"apiKey": "<opencode-cli virtual key>"
|
||||
},
|
||||
"models": {
|
||||
@@ -84,8 +84,8 @@ Select the model with `aiproxy/qwen3.8-27b-local`.
|
||||
|
||||
| CLI | Wire format | Endpoint | Config |
|
||||
|---|---|---|---|
|
||||
| Claude Code | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_API_PORT:-20129}` | `ANTHROPIC_BASE_URL` env var |
|
||||
| Kimi CLI | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_API_PORT:-20129}/v1` | `config.toml` provider block |
|
||||
| OpenCode | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_API_PORT:-20129}/v1` | `opencode.json` provider block |
|
||||
| Claude Code | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_PORT:-4000}` | `ANTHROPIC_BASE_URL` env var |
|
||||
| Kimi CLI | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `config.toml` provider block |
|
||||
| OpenCode | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `opencode.json` provider block |
|
||||
|
||||
Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/research/opencode-cli-setup.md`, `docs/proxy-key-onboarding.md`.
|
||||
|
||||
@@ -12,7 +12,7 @@ If you later want external CLI access too, that's a deliberate scope change —
|
||||
|
||||
As of [issue #31](https://git.arthurerlich.de/haylan/LLM-Server/issues/31) (migrated from LiteLLM), the gateway is OmniRoute:
|
||||
|
||||
- **`proxy.ai.home`** and **`proxy.ai.haylan.ch`** both point only at `${OMNIROUTE_API_PORT:-20129}` — the API port. Set up as two NPM Proxy Hosts pointing at this machine's LAN IP on that port; `ai.home` internal-only, `ai.haylan.ch` external via the DMZ already forwarding to NPM (let NPM issue/manage the TLS cert as usual).
|
||||
- **`proxy.ai.home`** and **`proxy.ai.haylan.ch`** both point only at `${OMNIROUTE_PORT:-4000}` — the API port. Set up as two NPM Proxy Hosts pointing at this machine's LAN IP on that port; `ai.home` internal-only, `ai.haylan.ch` external via the DMZ already forwarding to NPM (let NPM issue/manage the TLS cert as usual).
|
||||
- The **dashboard** (`${OMNIROUTE_DASHBOARD_PORT:-20128}`) is never registered in NPM at all, and `docker-compose.yml` never publishes that port to the host either — it manages every workload's keys, so it doesn't belong on the public internet, same reasoning as LiteLLM's old `/ui`. Unlike LiteLLM, OmniRoute's split-port mode means this is structural (no network route exists) rather than an NPM path-deny rule that has to be maintained and could be misconfigured. Reach the dashboard only from the host itself or over SSH port-forward.
|
||||
|
||||
**Every gateway call already requires a valid API key** (Bearer token, see `docs/proxy-key-onboarding.md`), so no extra NPM-level auth is needed for the external hostname.
|
||||
|
||||
@@ -6,7 +6,7 @@ No workload in this stack itself needs a key right now — every client is exter
|
||||
|
||||
## Create the key
|
||||
|
||||
1. Log into the omniroute dashboard (`http://<host>:${OMNIROUTE_DASHBOARD_PORT:-20128}` — LAN/host-only, never published publicly, see `docker-compose.yml`'s `omniroute` service).
|
||||
1. Log into the omniroute dashboard. `DASHBOARD_PORT` (20128) is never published to the host (see `docker-compose.yml`'s `omniroute` service) — from the R9700 box itself, find the container's own address (`docker inspect -f '{{.NetworkSettings.Networks.ai_stack.IPAddress}}' omniroute`) and browse to `http://<that-ip>:20128` (the host can reach a container's bridge-network IP directly, published port or not). From elsewhere, SSH port-forward instead: `ssh -L 20128:<container-ip>:20128 <host>`, then browse `http://localhost:20128`.
|
||||
2. "Keys" → "Create API key".
|
||||
3. Label it `<workload>-<purpose>` — a short slug matching the workload, e.g. `paperless-ocr`, `gitea-code-review`, `claude-code-cli`. This label is the ledger: the dashboard lists keys by label, so there's no separate tracking doc to keep in sync.
|
||||
4. Copy the key value shown — it's only shown once at creation, per OmniRoute's docs.
|
||||
|
||||
Reference in New Issue
Block a user