diff --git a/docs/research/opencode-cli-setup.md b/docs/research/opencode-cli-setup.md new file mode 100644 index 0000000..b52be38 --- /dev/null +++ b/docs/research/opencode-cli-setup.md @@ -0,0 +1,227 @@ +# OpenCode CLI: install + local llama.cpp provider config + +**Date:** 2026-08-24 +**Scope:** How to install OpenCode CLI and point it at this repo's llama.cpp OpenAI-compatible server +(`http://localhost:8080/v1`, serving Qwen3.8-27B) instead of a hosted provider. + +## 0. Which "opencode" is this? + +Confirmed project: **the OpenCode coding agent**, docs at https://opencode.ai/docs/, repo currently at +**github.com/anomalyco/opencode**. + +- The repo was **formerly `sst/opencode`** and was moved to `anomalyco/opencode` as part of the SST + team's org rebrand to "Anomaly." Multiple in-repo issues track fallout from the move (stale + `ghcr.io/sst/opencode` Docker tips, Homebrew tap migration, tool integrations still pointing at the + old path). Sources: + https://github.com/anomalyco/opencode/issues/8390 , + https://github.com/anomalyco/opencode/issues/6841 , + https://github.com/anomalyco/opencode/issues/16440 , + https://news.ycombinator.com/item?id=46552218 (community confirmation of the org move). + **Confidence: high** — corroborated by several independent primary-source issues in the same repo, + though I did not find an explicit "we renamed the org" announcement post to cite as the single + authoritative statement (the docs site itself just says "OpenCode" with no history section). +- **Naming collision to avoid**: `opencode-ai/opencode` (a different, unrelated Go/Bubble Tea TUI + project, MIT licensed, also calling itself "a powerful AI coding agent") turned up in search results + and is **not** this project. The ticket's target — `sst/opencode` — is the TypeScript/Bun project now + at `anomalyco/opencode`, confirmed by docs domain `opencode.ai` matching the install script URL used + by both the old and current repo. Do not install `opencode-ai/opencode` by mistake — check `opencode + --version` output / package name (`opencode-ai` on npm) if unsure. + Source: https://github.com/opencode-ai/opencode (surfaced in search, cross-checked and excluded). + +## 1. Install + +Official methods, from https://opencode.ai/docs/ and https://github.com/anomalyco/opencode README +(fetched directly): + +```bash +# curl install script (recommended by the docs) +curl -fsSL https://opencode.ai/install | bash + +# npm +npm i -g opencode-ai@latest + +# Homebrew +brew install anomalyco/tap/opencode +``` + +Also documented as available: Bun, pnpm, Yarn, Arch `pacman`/`paru`, Windows Chocolatey/Scoop, Mise, +Docker. Source: https://opencode.ai/docs/ (Installation section). + +**Confidence: high** — these are the exact commands returned by fetching the docs and GitHub README +directly, not a paraphrase from a third-party blog. + +## 2. Config file: location and format + +Format: **JSON or JSONC** (JSON with comments). File name: `opencode.json` (or `opencode.jsonc`). + +- **Global**: `~/.config/opencode/opencode.json` — user-wide settings. +- **Project**: `opencode.json` in the project root — picked up per-project, overrides/merges with + global. +- A separate `tui.json` (same locations) holds TUI-only settings (themes, keybinds) — not needed for + provider config. +- Schema for editor validation/autocomplete: `"$schema": "https://opencode.ai/config.json"`. + +Source: https://opencode.ai/docs/config/ (fetched directly). +**Confidence: high.** + +## 3. Declaring a local OpenAI-compatible provider (llama.cpp) + +Two-part setup per the docs: + +1. Optionally run `/connect` inside OpenCode, choose "Other", and give the provider a unique ID — this + only stores a credential (which can be left blank/dummy for a local no-auth server) and does **not** + generate the provider config block itself. +2. Hand-write the provider block in `opencode.json` (project or global). Exact structure from + https://opencode.ai/docs/providers/ (fetched directly, JSON quoted verbatim from the docs): + +```json +{ + "$schema": "https://opencode.ai/config.json", + "provider": { + "llamacpp": { + "npm": "@ai-sdk/openai-compatible", + "name": "llama.cpp (local)", + "options": { + "baseURL": "http://localhost:8080/v1", + "apiKey": "sk-local-not-checked" + }, + "models": { + "qwen3.8-27b": { + "name": "Qwen3.8-27B", + "limit": { + "context": 131072, + "output": 8192 + } + } + } + } + } +} +``` + +Key fields: +- **`npm`**: `"@ai-sdk/openai-compatible"` — the Vercel AI SDK adapter OpenCode uses for any server + speaking `/v1/chat/completions`. This is what makes it work against llama.cpp's OpenAI-compatible + endpoint (this repo's `/v1`), not the Anthropic shim (`/v1/messages` — see §4/§5 below for why the + OpenAI-compatible endpoint is the correct target, not the Anthropic shim, for OpenCode specifically). +- **`options.baseURL`**: the server URL up to and including `/v1` — for this repo, + `http://localhost:8080/v1`. +- **`options.apiKey`**: docs show it as *optional*, using `"{env:VAR_NAME}"` syntax to pull from an + env var in the general/hosted-provider example. For a local server that doesn't check the key, + supplying any non-empty string (or omitting it) both appear to work per the docs' own local-provider + example, which omits `apiKey` entirely for a bare local baseURL. Because the AI SDK's OpenAI-compatible + client sometimes still requires a non-empty `Authorization` header to be well-formed, the safer choice + is a dummy literal string (e.g. `"sk-local-not-checked"`) rather than omitting the field — + **this specific claim (dummy string vs. omit) is not directly confirmed by a docs quote either way**; + treat as a practical recommendation, not a documented requirement. **Confidence: medium** on the + omit-vs-dummy distinction; **high** on baseURL/npm/models structure itself. +- **`models.`**: the model ID key must match what the server exposes (check + `curl http://localhost:8080/v1/models`). `limit.context` / `limit.output` are optional token-limit + hints OpenCode uses for context-window bookkeeping — not enforced by the server itself. +- Then select the model in OpenCode with `/`, e.g. `llamacpp/qwen3.8-27b` + (format documented on the troubleshooting page: "Models should be referenced like so: + `/`"). Source: https://opencode.ai/docs/troubleshooting/. + +Source for the whole block: https://opencode.ai/docs/providers/ (fetched directly). +**Confidence: high** on the JSON shape; the exact llama.cpp `baseURL`/port is this repo's own +docker-compose (`8080`), not an assumption. + +## 4. Documented compatibility notes: self-hosted OpenAI-compatible servers generally + +Two relevant, **currently open/closed-not-planned** primary-source issues in `anomalyco/opencode`: + +- **Issue #20669** — "Default agent is brittle against local OpenAI-compatible tool-call quirks." + Documents two concrete failure modes against local backends (llama.cpp, LM Studio) in OpenCode's + agent loop: + 1. The `bash` tool throws a hard error ("expected string, received undefined") if the model emits a + valid `command` but omits the `description` field — some local models don't reliably fill optional + tool-call fields the way hosted models do. + 2. Some local backends return `finish_reason: "tool_calls"` together with an **empty** `tool_calls: []` + array; OpenCode's loop doesn't treat this as a normal stop and can loop/hang instead. + **Resolution: closed as "not planned"** (maintainers declined to add compatibility shims for this), + referencing PR #26419. No documented workaround from the maintainers. + Source: https://github.com/anomalyco/opencode/issues/20669 (fetched directly). + **Confidence: high** on the issue content; the "not planned" resolution means this is a **live, + acknowledged, unfixed risk** for any local OpenAI-compatible backend, not specific to llama.cpp. + +- **Issue #1890** — "OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error + unless `--jinja`; with `--jinja`, template crashes (reject filter)." llama.cpp-specific: + - OpenCode unconditionally includes `tools` scaffolding in the request even for plain-chat agents + with no tools configured. + - Without llama.cpp's `--jinja` server flag: llama.cpp rejects the request outright + (`"tools param requires --jinja flag"`). + - With `--jinja`: llama.cpp's Jinja renderer crashes on some models' chat templates + (`"Value is not callable: null"`, a `reject` filter llama.cpp's Jinja implementation doesn't + support) — a 500 error. + - Tested against OpenCode v0.4.40/v0.4.41 and a Qwen3-30B model. + - Workarounds discussed: run llama.cpp with `--jinja` and a minimal template that ignores tools, or + proxy requests to strip `tools`/`tool_choice` before they reach llama.cpp. + Source: https://github.com/anomalyco/opencode/issues/1890 (fetched directly). + **Confidence: high** on content; **medium** on current relevance to this repo's exact llama.cpp/model + version since the issue is against an older OpenCode build and doesn't confirm current-build behavior. + +**Practical implication for this repo**: this repo's llama.cpp container should be launched with +`--jinja` (check `docker-compose.yml` / entrypoint flags) or tool-calling via OpenCode will hard-fail +with a 500 immediately — and even with `--jinja`, Qwen-family chat templates have a documented history +of crashing llama.cpp's Jinja renderer (see also this repo's own +`docs/research/qwen3.8-27b-tool-calling.md`, which independently found llama.cpp's tool-call +parser/grammar for the Qwen3.5 lineage — which Qwen3.8-27B shares — to be broken in multiple +still-open llama.cpp upstream issues, unrelated to OpenCode). + +## 5. Model-specific notes for Qwen3.8-27B vs. what OpenCode expects from cloud models + +- **Tool-calling reliability is the dominant risk, not context/thinking-mode handling.** This repo's + own prior research (`docs/research/qwen3.8-27b-tool-calling.md`) independently concluded, via + llama.cpp upstream issues, that Qwen3.8-27B's architecture lineage (Qwen3.5, hybrid Gated + DeltaNet/attention) has **documented, still-open llama.cpp tool-call parser bugs** — this is a + llama.cpp-side problem that affects both the Anthropic shim and the plain OpenAI-compatible endpoint + identically, since the Anthropic shim just re-uses llama.cpp's normal chat-completions tool-calling + pipeline (source: https://github.com/ggml-org/llama.cpp/pull/17570, per that same file). Combined with + OpenCode's own §4 issues, expect tool-calling from OpenCode against this specific model to be + **unreliable until upstream llama.cpp fixes land**, independent of OpenCode config correctness. +- **Long context**: OpenCode's `limit.context`/`limit.output` fields (§3) are just bookkeeping hints for + OpenCode's own context-management UI/truncation logic — they don't change what the server actually + accepts. Set them to match whatever `--ctx-size` this repo's llama.cpp container is actually launched + with (check `docker-compose.yml`), not a value assumed from the model card, or OpenCode may + miscalculate when to compact/summarize the conversation. +- **Thinking/reasoning-mode handling**: OpenCode's documented reasoning-effort config + (`options.reasoningEffort`, `options.thinking.budgetTokens` — https://opencode.ai/docs/models/, + fetched directly) is written for providers with a **native reasoning-effort API parameter** (OpenAI's + `reasoningEffort`, Anthropic's `thinking` block) — i.e., the hosted provider itself understands the + knob and returns reasoning separately from the answer. **The docs do not document any handling for + models that instead emit inline `...` tags in the raw completion text** (the pattern + Qwen3-family "hybrid thinking" models typically use when reasoning isn't a first-class API field). + For a llama.cpp-served Qwen3.8-27B, `reasoningEffort`/`thinking` config keys almost certainly have + **no effect** (llama.cpp's OpenAI-compatible endpoint doesn't expose or consume those keys) — thinking + mode for this model would need to be controlled at the llama.cpp/prompt level (e.g., a + `/no_think` marker or template-level toggle, if Qwen3.8-27B's chat template supports one), and + whether OpenCode strips or garbles `` blocks in the response was **not found documented + anywhere in the primary sources checked**. **Confidence: low** on this specific sub-claim — flagged + as an open question, not a verified fact. Recommend testing empirically once tool-calling is unblocked. + +## Sources consulted (primary) + +- https://opencode.ai/docs/ — install methods, docs nav +- https://opencode.ai/docs/config/ — config file location/format +- https://opencode.ai/docs/providers/ — custom OpenAI-compatible provider JSON +- https://opencode.ai/docs/models/ — reasoning effort / thinking config +- https://opencode.ai/docs/troubleshooting/ — model ID reference format +- https://github.com/anomalyco/opencode — repo README, install commands, license +- https://github.com/anomalyco/opencode/issues/20669 — local-backend tool-call brittleness (closed, not planned) +- https://github.com/anomalyco/opencode/issues/1890 — llama.cpp `--jinja` / template crash +- https://github.com/anomalyco/opencode/issues/8390, #6841, #16440 — org rename (`sst` → `anomalyco`) fallout +- https://github.com/ggml-org/llama.cpp/pull/17570 — Anthropic Messages shim reuses OpenAI tool-calling pipeline (also cited in this repo's `docs/research/qwen3.8-27b-tool-calling.md`) +- This repo: `docs/research/qwen3.8-27b-tool-calling.md` — independent finding of open llama.cpp tool-call parser bugs for the Qwen3.5/Qwen3.8 lineage + +## Confidence summary + +| Claim | Confidence | +|---|---| +| Correct project = `anomalyco/opencode` (formerly `sst/opencode`) | High | +| Install commands | High | +| Config file location/format | High | +| Provider JSON shape (npm/baseURL/models) | High | +| apiKey omit-vs-dummy for no-auth servers | Medium (not directly confirmed) | +| OpenCode local-backend tool-call brittleness (#20669) | High (content); this is a live unfixed issue | +| llama.cpp `--jinja` requirement / template crash (#1890) | High (content); Medium (currency vs. latest builds) | +| Qwen3.8-27B thinking-tag handling in OpenCode | Low — undocumented, flagged as open question |