docs: restructure around source-of-truth pages

- CONTEXT.md glossary
- README reduced to intro, quickstart and doc index
- new architecture.md, gateway.md (from proxy-key-onboarding.md),
  operations.md; network-access.md rewritten
- coding-cli-setup: stale OMNIROUTE_PORT:-4000 endpoint, terms and
  duplicated per-slot/classifier explanations fixed
- research/README.md index; proxy-request-priority.md moved there as
  superseded

Every doc names keys instead of restating values and links its source.

Refs #55 (#56, #57, #59, #60)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
haylanandClaude Bot committed 2026-09-24 21:51:01 +02:00
1 parent 072eeb78c2
commit a0e5e9704b
14 files changed
+340 -119

No files matched your search

+17 -22
View File
@@ -1,34 +1,29 @@
# Pointing a coding-agent CLI at this stack
# Pointing a coding CLI at this stack
[← back to README](../../README.md)
This stack routes through the [AI gateway](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) (OmniRoute, see issue #31) rather than talking to llama.cpp directly — llama.cpp's own port is internal-only now (see `docker-compose.yml`). The gateway exposes:
> **Source of truth:** each CLI's own config docs (linked per page), plus [`docker-compose.yml`](../../docker-compose.yml) → `omniroute` for the gateway port. This page explains *how to wire them up*; ports and model settings live in those sources. If they disagree, the sources win.
- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1`
- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_PORT:-4000}`
Terms as defined in [`CONTEXT.md`](../../CONTEXT.md). Every coding CLI is a workload: it reaches the models through the gateway, never a backend directly ([network access](../network-access.md)). The gateway speaks two wire formats for the same models:
Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, registered in the gateway (naming is yours to pick when adding the llama-cpp provider connection — these docs assume `qwen3.8-27b-local` for continuity) — behind whichever wire format the client speaks.
- **OpenAI-compatible:** `http://<server>:<OMNIROUTE_API_PORT>/v1`
- **Anthropic Messages:** `http://<server>:<OMNIROUTE_API_PORT>` (OmniRoute's `/v1/messages`)
`<ai-box>` is this machine's LAN address, or `proxy-ai.home` if your local DNS resolves that hostname directly to the box — see `docs/network-access.md`. If you're running a coding CLI from this machine itself, `localhost` works too.
`<server>` is the server's LAN address or `proxy-ai.home`, or `localhost` on the server itself. The main model's name is whatever it was registered as in the gateway. These docs use `qwen3.8-27b-local` ([gateway → Provider connections](../gateway.md#provider-connections)).
**Each CLI needs its own virtual key** — create one per docs/proxy-key-onboarding.md (omniroute's dashboard, `<workload>-<purpose>` naming, e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli`). No budget set by default. These are the machine's interactive/high-priority workloads per `docs/proxy-request-priority.md`.
**Each CLI needs its own API key**, labelled e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli` ([gateway → API keys](../gateway.md#api-keys)).
> **Read this before relying on it for real work.** Qwen3.8-27B's tool-calling has **documented, open llama.cpp upstream bugs** (parser fails on text before `<tool_call>`, tool calls emitted as inert XML inside thinking blocks — see `docs/research/qwen3.8-27b-tool-calling.md`). Every CLI below inherits this risk identically, regardless of wire format. Don't trust it for unattended multi-step agentic work until you've run the smoke test in [issue #5](https://git.arthurerlich.de/haylan/LLM-Server/issues/5) (and the proxy-specific smoke test in [issue #17](https://git.arthurerlich.de/haylan/LLM-Server/issues/17)).
**Context window:** wherever a CLI asks for the main model's context size, use the per-slot context, `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from the server's `.env`, not the raw context size ([architecture → llama-server](../architecture.md#llama-server)).
> **Tool-calling risk.** The main model's tool calling has open llama.cpp upstream parser bugs ([`research/qwen3.8-27b-tool-calling.md`](../research/qwen3.8-27b-tool-calling.md)). Every CLI below inherits that risk, whatever its wire format. Test it on your own workflow before relying on it for unattended multi-step agent work.
## Per-CLI setup
- [Claude Code CLI](claude-code.md)
- [Kimi CLI](kimi-cli.md)
- [OpenCode CLI](opencode.md)
- [Qwen Code CLI](qwen-code.md)
| CLI | Wire format | Config |
|---|---|---|
| [Claude Code](claude-code.md) | Anthropic Messages | `ANTHROPIC_BASE_URL` env var |
| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `config.toml` provider block |
| [OpenCode](opencode.md) | OpenAI Chat Completions | `opencode.json` provider block |
| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (main model + classifier model) | `~/.qwen/settings.json` `modelProviders.openai` |
## Summary
| CLI | Wire format | Endpoint | Config |
|---|---|---|---|
| [Claude Code](claude-code.md) | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_PORT:-4000}` | `ANTHROPIC_BASE_URL` env var |
| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `config.toml` provider block |
| [OpenCode](opencode.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `opencode.json` provider block |
| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (2 models: chat + `fastModel`) | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `~/.qwen/settings.json` `modelProviders.openai` |
Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/proxy-key-onboarding.md`, `docs/research/omniroute-account-semaphore-timeout.md` (a connection that can only handle a few concurrent requests — like `llama-server` or `qwen-classifier` — hits a hardcoded 30s reject once more requests queue up than its `maxConcurrent`, unless configured around it).
If requests fail with a semaphore timeout under concurrent load, see [gateway → Timeouts](../gateway.md#timeouts).