haylanandClaude Bot
17747694bd
Replace MTP with draft-dflash and a separate DFlash2 drafter GGUF. Remove the qwen-classifier backend to free VRAM (GPU was at 100%). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
30 lines
2.3 KiB
Markdown
30 lines
2.3 KiB
Markdown
# Pointing a coding CLI at this stack
|
|
|
|
[← back to README](../../README.md)
|
|
|
|
> **Source of truth:** each CLI's own config docs (linked per page), plus [`docker-compose.yml`](../../docker-compose.yml) → `omniroute` for the gateway port. This page explains *how to wire them up*; ports and model settings live in those sources. If they disagree, the sources win.
|
|
|
|
Terms as defined in [`CONTEXT.md`](../../CONTEXT.md). Every coding CLI is a workload: it reaches the models through the gateway, never a backend directly ([network access](../network-access.md)). The gateway speaks two wire formats for the same models:
|
|
|
|
- **OpenAI-compatible:** `http://<server>:<OMNIROUTE_API_PORT>/v1`
|
|
- **Anthropic Messages:** `http://<server>:<OMNIROUTE_API_PORT>` (OmniRoute's `/v1/messages`)
|
|
|
|
`<server>` is the server's LAN address or `proxy-ai.home`, or `localhost` on the server itself. The main model's name is whatever it was registered as in the gateway. These docs use `qwen3.8-27b-local` ([gateway → Provider connections](../gateway.md#provider-connections)).
|
|
|
|
**Each CLI needs its own API key**, labelled e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli` ([gateway → API keys](../gateway.md#api-keys)).
|
|
|
|
**Context window:** wherever a CLI asks for the main model's context size, use the per-slot context, `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from the server's `.env`, not the raw context size ([architecture → llama-server](../architecture.md#llama-server)).
|
|
|
|
> **Tool-calling risk.** The main model's tool calling has open llama.cpp upstream parser bugs ([`research/qwen3.8-27b-tool-calling.md`](../research/qwen3.8-27b-tool-calling.md)). Every CLI below inherits that risk, whatever its wire format. Test it on your own workflow before relying on it for unattended multi-step agent work.
|
|
|
|
## Per-CLI setup
|
|
|
|
| CLI | Wire format | Config |
|
|
|---|---|---|
|
|
| [Claude Code](claude-code.md) | Anthropic Messages | `ANTHROPIC_BASE_URL` env var |
|
|
| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `config.toml` provider block |
|
|
| [OpenCode](opencode.md) | OpenAI Chat Completions | `opencode.json` provider block |
|
|
| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (main model) | `~/.qwen/settings.json` `modelProviders.openai` |
|
|
|
|
If requests fail with a semaphore timeout under concurrent load, see [gateway → Timeouts](../gateway.md#timeouts).
|