docs: restructure around source-of-truth pages
- CONTEXT.md glossary - README reduced to intro, quickstart and doc index - new architecture.md, gateway.md (from proxy-key-onboarding.md), operations.md; network-access.md rewritten - coding-cli-setup: stale OMNIROUTE_PORT:-4000 endpoint, terms and duplicated per-slot/classifier explanations fixed - research/README.md index; proxy-request-priority.md moved there as superseded Every doc names keys instead of restating values and links its source. Refs #55 (#56, #57, #59, #60) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
072eeb78c2
commit
a0e5e9704b
14 files changed
+340
-119
No files matched your search
@@ -2,12 +2,14 @@
|
||||
|
||||
[← back to overview](index.md)
|
||||
|
||||
> **Source of truth:** [Claude Code's docs](https://docs.claude.com/en/docs/claude-code/overview) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
|
||||
|
||||
Claude Code speaks the **Anthropic Messages API** — point it at the gateway's unified endpoint, not llama.cpp directly:
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_BASE_URL=http://<ai-box>:${OMNIROUTE_PORT:-4000}
|
||||
export ANTHROPIC_API_KEY=<claude-code-cli virtual key>
|
||||
export ANTHROPIC_BASE_URL=http://<server>:<OMNIROUTE_API_PORT>
|
||||
export ANTHROPIC_API_KEY=<claude-code-cli API key>
|
||||
claude
|
||||
```
|
||||
|
||||
Requires llama.cpp's `--jinja` flag (already set in `docker-compose.yml`) — without it, tool-use requests fail outright.
|
||||
Relies on the backend's `--jinja` flag ([architecture → llama-server](../architecture.md#llama-server)).
|
||||
@@ -1,34 +1,29 @@
|
||||
# Pointing a coding-agent CLI at this stack
|
||||
# Pointing a coding CLI at this stack
|
||||
|
||||
[← back to README](../../README.md)
|
||||
|
||||
This stack routes through the [AI gateway](https://git.arthurerlich.de/haylan/LLM-Server/issues/9) (OmniRoute, see issue #31) rather than talking to llama.cpp directly — llama.cpp's own port is internal-only now (see `docker-compose.yml`). The gateway exposes:
|
||||
> **Source of truth:** each CLI's own config docs (linked per page), plus [`docker-compose.yml`](../../docker-compose.yml) → `omniroute` for the gateway port. This page explains *how to wire them up*; ports and model settings live in those sources. If they disagree, the sources win.
|
||||
|
||||
- **OpenAI-compatible**: `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1`
|
||||
- **Anthropic Messages API** (OmniRoute's own `/v1/messages` endpoint, translating to the OpenAI-compatible backend): `http://<ai-box>:${OMNIROUTE_PORT:-4000}`
|
||||
Terms as defined in [`CONTEXT.md`](../../CONTEXT.md). Every coding CLI is a workload: it reaches the models through the gateway, never a backend directly ([network access](../network-access.md)). The gateway speaks two wire formats for the same models:
|
||||
|
||||
Both serve the same underlying model — `Qwen3.8-27B-UD-Q4_K_XL.gguf`, registered in the gateway (naming is yours to pick when adding the llama-cpp provider connection — these docs assume `qwen3.8-27b-local` for continuity) — behind whichever wire format the client speaks.
|
||||
- **OpenAI-compatible:** `http://<server>:<OMNIROUTE_API_PORT>/v1`
|
||||
- **Anthropic Messages:** `http://<server>:<OMNIROUTE_API_PORT>` (OmniRoute's `/v1/messages`)
|
||||
|
||||
`<ai-box>` is this machine's LAN address, or `proxy-ai.home` if your local DNS resolves that hostname directly to the box — see `docs/network-access.md`. If you're running a coding CLI from this machine itself, `localhost` works too.
|
||||
`<server>` is the server's LAN address or `proxy-ai.home`, or `localhost` on the server itself. The main model's name is whatever it was registered as in the gateway. These docs use `qwen3.8-27b-local` ([gateway → Provider connections](../gateway.md#provider-connections)).
|
||||
|
||||
**Each CLI needs its own virtual key** — create one per docs/proxy-key-onboarding.md (omniroute's dashboard, `<workload>-<purpose>` naming, e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli`). No budget set by default. These are the machine's interactive/high-priority workloads per `docs/proxy-request-priority.md`.
|
||||
**Each CLI needs its own API key**, labelled e.g. `claude-code-cli`, `kimi-cli`, `opencode-cli` ([gateway → API keys](../gateway.md#api-keys)).
|
||||
|
||||
> **Read this before relying on it for real work.** Qwen3.8-27B's tool-calling has **documented, open llama.cpp upstream bugs** (parser fails on text before `<tool_call>`, tool calls emitted as inert XML inside thinking blocks — see `docs/research/qwen3.8-27b-tool-calling.md`). Every CLI below inherits this risk identically, regardless of wire format. Don't trust it for unattended multi-step agentic work until you've run the smoke test in [issue #5](https://git.arthurerlich.de/haylan/LLM-Server/issues/5) (and the proxy-specific smoke test in [issue #17](https://git.arthurerlich.de/haylan/LLM-Server/issues/17)).
|
||||
**Context window:** wherever a CLI asks for the main model's context size, use the per-slot context, `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from the server's `.env`, not the raw context size ([architecture → llama-server](../architecture.md#llama-server)).
|
||||
|
||||
> **Tool-calling risk.** The main model's tool calling has open llama.cpp upstream parser bugs ([`research/qwen3.8-27b-tool-calling.md`](../research/qwen3.8-27b-tool-calling.md)). Every CLI below inherits that risk, whatever its wire format. Test it on your own workflow before relying on it for unattended multi-step agent work.
|
||||
|
||||
## Per-CLI setup
|
||||
|
||||
- [Claude Code CLI](claude-code.md)
|
||||
- [Kimi CLI](kimi-cli.md)
|
||||
- [OpenCode CLI](opencode.md)
|
||||
- [Qwen Code CLI](qwen-code.md)
|
||||
| CLI | Wire format | Config |
|
||||
|---|---|---|
|
||||
| [Claude Code](claude-code.md) | Anthropic Messages | `ANTHROPIC_BASE_URL` env var |
|
||||
| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `config.toml` provider block |
|
||||
| [OpenCode](opencode.md) | OpenAI Chat Completions | `opencode.json` provider block |
|
||||
| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (main model + classifier model) | `~/.qwen/settings.json` `modelProviders.openai` |
|
||||
|
||||
## Summary
|
||||
|
||||
| CLI | Wire format | Endpoint | Config |
|
||||
|---|---|---|---|
|
||||
| [Claude Code](claude-code.md) | Anthropic Messages | `http://<ai-box>:${OMNIROUTE_PORT:-4000}` | `ANTHROPIC_BASE_URL` env var |
|
||||
| [Kimi CLI](kimi-cli.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `config.toml` provider block |
|
||||
| [OpenCode](opencode.md) | OpenAI Chat Completions | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `opencode.json` provider block |
|
||||
| [Qwen Code](qwen-code.md) | OpenAI Chat Completions (2 models: chat + `fastModel`) | `http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1` | `~/.qwen/settings.json` `modelProviders.openai` |
|
||||
|
||||
Further reading: `docs/research/qwen3.8-27b-tool-calling.md`, `docs/proxy-key-onboarding.md`, `docs/research/omniroute-account-semaphore-timeout.md` (a connection that can only handle a few concurrent requests — like `llama-server` or `qwen-classifier` — hits a hardcoded 30s reject once more requests queue up than its `maxConcurrent`, unless configured around it).
|
||||
If requests fail with a semaphore timeout under concurrent load, see [gateway → Timeouts](../gateway.md#timeouts).
|
||||
@@ -2,13 +2,15 @@
|
||||
|
||||
[← back to overview](index.md)
|
||||
|
||||
> **Source of truth:** [Kimi CLI's repo](https://github.com/MoonshotAI/kimi-cli) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
|
||||
|
||||
Kimi CLI speaks plain **OpenAI Chat Completions**. Configure a provider block in its config file (`config.toml`):
|
||||
|
||||
```toml
|
||||
[providers.openai]
|
||||
type = "openai"
|
||||
base_url = "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
|
||||
api_key = "<kimi-cli virtual key>"
|
||||
base_url = "http://<server>:<OMNIROUTE_API_PORT>/v1"
|
||||
api_key = "<kimi-cli API key>"
|
||||
```
|
||||
|
||||
If Kimi CLI's response parsing gets confused by Qwen's `<think>...</think>` reasoning tags, check its `reasoning_key` setting — it's configurable for non-standard local server responses.
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
[← back to overview](index.md)
|
||||
|
||||
> **Source of truth:** [OpenCode's docs](https://opencode.ai/docs/) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
|
||||
|
||||
Confirmed project: **`anomalyco/opencode`** (renamed from `sst/opencode` — don't confuse with the unrelated `opencode-ai/opencode` Go TUI). Docs: https://opencode.ai/docs/
|
||||
|
||||
**Install**:
|
||||
@@ -15,17 +17,17 @@ curl -fsSL https://opencode.ai/install | bash
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"aiproxy": {
|
||||
"llm-server": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "AI proxy (local)",
|
||||
"name": "LLM-Server gateway",
|
||||
"options": {
|
||||
"baseURL": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
|
||||
"apiKey": "<opencode-cli virtual key>"
|
||||
"baseURL": "http://<server>:<OMNIROUTE_API_PORT>/v1",
|
||||
"apiKey": "<opencode-cli API key>"
|
||||
},
|
||||
"models": {
|
||||
"qwen3.8-27b-local": {
|
||||
"name": "Qwen3.8-27B",
|
||||
"limit": { "context": 131072, "output": 8192 }
|
||||
"limit": { "context": <LLAMA_CTX_SIZE / LLAMA_PARALLEL>, "output": 8192 }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -33,13 +35,13 @@ curl -fsSL https://opencode.ai/install | bash
|
||||
}
|
||||
```
|
||||
|
||||
Set `limit.context` to the *per-slot* context this stack actually serves — `LLAMA_CTX_SIZE / LLAMA_PARALLEL` from `.env` (262144 / 2 = 131072 by default), not raw `LLAMA_CTX_SIZE` and not a value assumed from the model card: llama.cpp divides `--ctx-size` across concurrent slots, so each request only gets one slot's share. OpenCode uses this for its own context-management bookkeeping, not the server.
|
||||
`limit.context` is the per-slot context ([overview](index.md)). OpenCode uses it for its own context bookkeeping; the server doesn't read it.
|
||||
|
||||
Select the model with `aiproxy/qwen3.8-27b-local`.
|
||||
Select the model with `llm-server/qwen3.8-27b-local`.
|
||||
|
||||
**OpenCode-specific risks** (on top of the shared Qwen3.8-27B tool-calling risk — see [overview](index.md)):
|
||||
- Requires llama.cpp's `--jinja` flag (already set) — without it, OpenCode's unconditional tool-call scaffolding gets a 500.
|
||||
- Relies on the backend's `--jinja` flag ([architecture → llama-server](../architecture.md#llama-server)); without it, OpenCode's unconditional tool-call scaffolding gets a 500.
|
||||
- [anomalyco/opencode#20669](https://github.com/anomalyco/opencode/issues/20669) (closed as "not planned" — a live, unfixed risk): OpenCode's `bash` tool crashes if the model omits the optional `description` field on a tool call; some local backends return `finish_reason: tool_calls` with an empty array, which can hang the agent loop instead of stopping cleanly.
|
||||
- Thinking-mode handling (`options.reasoningEffort`) is undocumented for models that emit inline `<think>` tags rather than a native reasoning API field — expect no effect from that config on this model; untested.
|
||||
|
||||
Further reading: `docs/research/opencode-cli-setup.md`.
|
||||
Further reading: [`research/opencode-cli-setup.md`](../research/opencode-cli-setup.md) (its base URL example predates the gateway), [`research/opencode-auto-compact.md`](../research/opencode-auto-compact.md).
|
||||
@@ -2,16 +2,13 @@
|
||||
|
||||
[← back to overview](index.md)
|
||||
|
||||
Qwen Code speaks plain **OpenAI Chat Completions**, and — unlike the other CLIs — needs *two* models: the main chat model, and a `fastModel` for Auto Mode's action classifier. Both are registered as separate providers in OmniRoute but reachable through the same gateway URL.
|
||||
> **Source of truth:** [Qwen Code's repo](https://github.com/QwenLM/qwen-code) for config keys; the [overview](index.md) for the gateway endpoint and context size. If this page disagrees with them, they win.
|
||||
|
||||
Qwen Code speaks plain **OpenAI Chat Completions**. Unlike the other CLIs, it needs *two* models: the main model, and the classifier model as its `fastModel` for Auto Mode's action classifier. Each has its own provider connection in the gateway, and both are reached through the same gateway URL.
|
||||
|
||||
## Why a second model exists
|
||||
|
||||
Auto Mode's action classifier (`permissions.autoMode`) is qwen-code's per-tool-call safety gate — it decides whether to auto-approve or block a shell command / tool call before it runs. It was originally aliased onto the main 27B model's own OmniRoute connection. That broke two ways in practice (see `docs/research/fast-model-choice.md` for the model research, and the issue-tracker history for the full incident):
|
||||
|
||||
- **Queued behind heavy work.** Every classification call competed for the main model's 2 GPU slots with whatever real generation was already running, so a classifier check could sit blocked for minutes.
|
||||
- **CPU-only was tried first and was too slow.** Isolating the classifier onto its own CPU-only llama.cpp instance avoided the GPU queue entirely, but real classification calls (which can carry a non-trivial conversation transcript, not just the bare tool call) blew past OmniRoute's request timeout and retry-looped.
|
||||
|
||||
The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.yml`) running a small model (`Qwen3-4B-Instruct-2507`) on its own **partial** GPU offload — enough layers on the R9700 to be fast, sized to leave real VRAM headroom next to the 27B model rather than trusting a naive weights+KV estimate (see that service's comment block in `docker-compose.yml` for the actual measured numbers and the two wrong turns — batch-size tuning, then flash-attn — before partial offload turned out to be the real lever).
|
||||
Auto Mode's action classifier (`permissions.autoMode`) is qwen-code's safety gate for every tool call: it decides whether to auto-approve or block a shell command or tool call before it runs. It runs on its own backend, `qwen-classifier`. Why that backend exists and how it's sized: [architecture → qwen-classifier](../architecture.md#qwen-classifier).
|
||||
|
||||
## `~/.qwen/settings.json`
|
||||
|
||||
@@ -23,16 +20,16 @@ The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.ym
|
||||
"id": "<main-model-provider-id-in-omniroute>",
|
||||
"name": "qwen3.8-27b-local",
|
||||
"envKey": "OMNIROUTE_API_KEY",
|
||||
"baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
|
||||
"generationConfig": { "contextWindowSize": 131072 }
|
||||
"baseUrl": "http://<server>:<OMNIROUTE_API_PORT>/v1",
|
||||
"generationConfig": { "contextWindowSize": <LLAMA_CTX_SIZE / LLAMA_PARALLEL> }
|
||||
},
|
||||
{
|
||||
"id": "<classifier-provider-id-in-omniroute>",
|
||||
"name": "qwen3-4b-classifier",
|
||||
"envKey": "OMNIROUTE_API_KEY",
|
||||
"baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1",
|
||||
"baseUrl": "http://<server>:<OMNIROUTE_API_PORT>/v1",
|
||||
"generationConfig": {
|
||||
"contextWindowSize": 65536,
|
||||
"contextWindowSize": <qwen-classifier --ctx-size>,
|
||||
"extra_body": { "chat_template_kwargs": { "enable_thinking": false } }
|
||||
}
|
||||
}
|
||||
@@ -41,21 +38,21 @@ The fix: a dedicated, GPU-resident `qwen-classifier` service (`docker-compose.ym
|
||||
"security": { "auth": { "selectedType": "openai" } },
|
||||
"model": {
|
||||
"name": "<main-model-provider-id-in-omniroute>",
|
||||
"baseUrl": "http://<ai-box>:${OMNIROUTE_PORT:-4000}/v1"
|
||||
"baseUrl": "http://<server>:<OMNIROUTE_API_PORT>/v1"
|
||||
},
|
||||
"fastModel": "<classifier-provider-id-in-omniroute>"
|
||||
}
|
||||
```
|
||||
|
||||
- `envKey` names the environment variable Qwen Code reads the virtual key from — set `OMNIROUTE_API_KEY=<qwen-code-cli virtual key>` before launching. Both providers can share one virtual key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls.
|
||||
- **`contextWindowSize` for the main model is per-slot, not `LLAMA_CTX_SIZE` itself** — llama.cpp divides `--ctx-size` across `LLAMA_PARALLEL` concurrent slots, and each request only gets one slot's share (same correction applies to OpenCode's `limit.context`). Compute it from `.env`: `LLAMA_CTX_SIZE / LLAMA_PARALLEL` = `262144 / 2` = **131072**.
|
||||
- **The classifier's `contextWindowSize` (65536) is not per-slot math** — `qwen-classifier` runs `--parallel 1`, so its whole `--ctx-size` belongs to the one slot. 65536 isn't a guess either: qwen-code's own source hard-caps the classifier transcript (`MAX_TRANSCRIPT_MESSAGES=40`, `MAX_HISTORICAL_ACTION_CHARS=4000`/message in `packages/core/src/permissions/classifier-transcript.ts`) — worst case is ~40-50K tokens, so 65536 gives real margin without wasting VRAM the way the original 131072 (copied from the main model's entry, not an actual qwen-code requirement) would have.
|
||||
- `enable_thinking: false` on the classifier matters for parseability, though `Qwen3-4B-Instruct-2507` is already architecturally non-thinking (see `fast-model-choice.md` §3) — this is belt-and-suspenders for any future fast-model swap that isn't.
|
||||
- `envKey` names the environment variable Qwen Code reads the API key from — set `OMNIROUTE_API_KEY=<qwen-code-cli API key>` before launching. Both providers can share one API key (as above); split it into two if you want separate usage tracking for chat vs. classifier calls.
|
||||
- **The main model's `contextWindowSize` is the per-slot context**, not `LLAMA_CTX_SIZE` itself ([overview](index.md)).
|
||||
- **The classifier's `contextWindowSize` is its whole `--ctx-size`.** It has a single slot, so there's no per-slot division ([architecture → qwen-classifier](../architecture.md#qwen-classifier)).
|
||||
- `enable_thinking: false` on the classifier matters for parseability, though `Qwen3-4B-Instruct-2507` is already architecturally non-thinking (see [`research/fast-model-choice.md`](../research/fast-model-choice.md) §3) — this is belt-and-suspenders for any future fast-model swap that isn't.
|
||||
- Qwen Code also recognizes `advisorModel`, `visionModel`, `compactionModel`, `imageModel` for other model roles — none are wired up in this stack; only `fastModel` is required.
|
||||
|
||||
## Web search via OmniRoute
|
||||
|
||||
Qwen Code's own built-in web search (`tools.webSearch.enabled`) has nothing to search with here — leave it `false`. Instead this stack's SearXNG-backed search (README §"Web search") is exposed through a thin stdio MCP wrapper around OmniRoute's `/v1/search` REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. `~/.qwen/mcp-servers/omniroute-search/index.mjs` (needs `@modelcontextprotocol/sdk` and `zod`: `npm init -y && npm i @modelcontextprotocol/sdk zod` in that directory):
|
||||
Qwen Code's own built-in web search (`tools.webSearch.enabled`) has nothing to search with here — leave it `false`. Instead this stack's SearXNG-backed search ([gateway → Web search](../gateway.md#web-search)) is exposed through a thin stdio MCP wrapper around OmniRoute's `/v1/search` REST endpoint (that endpoint isn't itself MCP — OmniRoute's real MCP surface is admin-only/LOCAL_ONLY-gated). Save this as e.g. `~/.qwen/mcp-servers/omniroute-search/index.mjs` (needs `@modelcontextprotocol/sdk` and `zod`: `npm init -y && npm i @modelcontextprotocol/sdk zod` in that directory):
|
||||
|
||||
```js
|
||||
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
||||
@@ -104,13 +101,13 @@ Register it in `~/.qwen/settings.json`:
|
||||
}
|
||||
```
|
||||
|
||||
It reuses the same `OMNIROUTE_API_KEY` env var as the model providers above — the virtual key needs search permission in OmniRoute, not just chat-completions.
|
||||
It reuses the same `OMNIROUTE_API_KEY` env var as the model providers above — the API key needs search permission in OmniRoute, not just chat-completions.
|
||||
|
||||
**Non-interactive mode (`qwen -p ...`) needs this tool explicitly allow-listed.** MCP tools require interactive confirmation by default; `--approval-mode auto` alone doesn't bypass that for a non-interactive run — pass `--allowed-tools mcp__omniroute-search__search` (or `-y` for full YOLO) alongside `-p`, or the search call never reaches the classifier at all and silently no-ops. Confirmed live: without the allow-list, only the tool calls the CLI's non-interactive gate lets through end up as classifier requests.
|
||||
|
||||
## Auto Mode tuning
|
||||
|
||||
Auto Mode's action classifier calls the fast model above. Even on the dedicated GPU-resident instance, give it real timeout headroom rather than trusting OmniRoute's default — and since this stack is a single trusted local proxy, it's reasonable to pre-approve requests to it rather than confirm every call:
|
||||
Auto Mode's action classifier calls the classifier model above. Even on its own GPU-resident backend, give it real timeout headroom rather than trusting OmniRoute's default. And since the gateway is a single trusted local endpoint, it's reasonable to pre-approve requests to it rather than confirm every call:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -125,6 +122,6 @@ Auto Mode's action classifier calls the fast model above. Even on the dedicated
|
||||
|
||||
`hints.allow` entries are free-text descriptions the classifier matches against, not exact strings — capped at 150 entries/200 chars each.
|
||||
|
||||
Also set a generous per-connection timeout on the classifier's own OmniRoute provider connection (`providerSpecificData.timeoutMs`, dashboard or `PATCH /api/providers/{id}` — not a `.env` value, see `docs/network-access.md` for reaching the dashboard API). 120000ms is comfortable for the current GPU-resident setup (real measured latency: well under a second for a short check, low seconds for the largest realistic transcript) — this isn't the 20-minute figure the main 27B connection needs, since the classifier isn't competing for a contended GPU slot the way the main model can.
|
||||
Also set a generous per-connection timeout on the classifier's own OmniRoute provider connection (`providerSpecificData.timeoutMs`, dashboard or `PATCH /api/providers/{id}` — not a `.env` value, see [gateway → Dashboard](../gateway.md#dashboard)). 120000ms is comfortable for the current GPU-resident setup (real measured latency: well under a second for a short check, low seconds for the largest realistic transcript) — this isn't the 20-minute figure the main 27B connection needs, since the classifier isn't competing for a contended GPU slot the way the main model can.
|
||||
|
||||
Everything else in `~/.qwen/settings.json` (`hooks`, `security.auth`'s underlying tooling, editor prefs) is per-machine, not part of pointing at this stack — don't copy it wholesale between machines.
|
||||
Reference in new issue
Block a user