research: OpenCode CLI install + local llama.cpp provider config

Answers issue #7. Confirms sst/opencode moved to anomalyco/opencode, documents install methods, opencode.json provider config for a local OpenAI-compatible server, and known tool-calling compatibility issues against self-hosted backends (llama.cpp --jinja requirement, empty tool_calls handling) plus Qwen3.8-27B-specific caveats. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 18:17:14 +02:00
parent 408a1f16ae
commit b773d9cb8f
1 changed files with 227 additions and 0 deletions
@@ -0,0 +1,227 @@
+# OpenCode CLI: install + local llama.cpp provider config
+
+**Date:** 2026-08-24
+**Scope:** How to install OpenCode CLI and point it at this repo's llama.cpp OpenAI-compatible server
+(`http://localhost:8080/v1`, serving Qwen3.8-27B) instead of a hosted provider.
+
+## 0. Which "opencode" is this?
+
+Confirmed project: **the OpenCode coding agent**, docs at https://opencode.ai/docs/, repo currently at
+**github.com/anomalyco/opencode**.
+
+- The repo was **formerly `sst/opencode`** and was moved to `anomalyco/opencode` as part of the SST
+  team's org rebrand to "Anomaly." Multiple in-repo issues track fallout from the move (stale
+  `ghcr.io/sst/opencode` Docker tips, Homebrew tap migration, tool integrations still pointing at the
+  old path). Sources:
+  https://github.com/anomalyco/opencode/issues/8390 ,
+  https://github.com/anomalyco/opencode/issues/6841 ,
+  https://github.com/anomalyco/opencode/issues/16440 ,
+  https://news.ycombinator.com/item?id=46552218 (community confirmation of the org move).
+  **Confidence: high** — corroborated by several independent primary-source issues in the same repo,
+  though I did not find an explicit "we renamed the org" announcement post to cite as the single
+  authoritative statement (the docs site itself just says "OpenCode" with no history section).
+- **Naming collision to avoid**: `opencode-ai/opencode` (a different, unrelated Go/Bubble Tea TUI
+  project, MIT licensed, also calling itself "a powerful AI coding agent") turned up in search results
+  and is **not** this project. The ticket's target — `sst/opencode` — is the TypeScript/Bun project now
+  at `anomalyco/opencode`, confirmed by docs domain `opencode.ai` matching the install script URL used
+  by both the old and current repo. Do not install `opencode-ai/opencode` by mistake — check `opencode
+  --version` output / package name (`opencode-ai` on npm) if unsure.
+  Source: https://github.com/opencode-ai/opencode (surfaced in search, cross-checked and excluded).
+
+## 1. Install
+
+Official methods, from https://opencode.ai/docs/ and https://github.com/anomalyco/opencode README
+(fetched directly):
+
+```bash
+# curl install script (recommended by the docs)
+curl -fsSL https://opencode.ai/install | bash
+
+# npm
+npm i -g opencode-ai@latest
+
+# Homebrew
+brew install anomalyco/tap/opencode
+```
+
+Also documented as available: Bun, pnpm, Yarn, Arch `pacman`/`paru`, Windows Chocolatey/Scoop, Mise,
+Docker. Source: https://opencode.ai/docs/ (Installation section).
+
+**Confidence: high** — these are the exact commands returned by fetching the docs and GitHub README
+directly, not a paraphrase from a third-party blog.
+
+## 2. Config file: location and format
+
+Format: **JSON or JSONC** (JSON with comments). File name: `opencode.json` (or `opencode.jsonc`).
+
+- **Global**: `~/.config/opencode/opencode.json` — user-wide settings.
+- **Project**: `opencode.json` in the project root — picked up per-project, overrides/merges with
+  global.
+- A separate `tui.json` (same locations) holds TUI-only settings (themes, keybinds) — not needed for
+  provider config.
+- Schema for editor validation/autocomplete: `"$schema": "https://opencode.ai/config.json"`.
+
+Source: https://opencode.ai/docs/config/ (fetched directly).
+**Confidence: high.**
+
+## 3. Declaring a local OpenAI-compatible provider (llama.cpp)
+
+Two-part setup per the docs:
+
+1. Optionally run `/connect` inside OpenCode, choose "Other", and give the provider a unique ID — this
+   only stores a credential (which can be left blank/dummy for a local no-auth server) and does **not**
+   generate the provider config block itself.
+2. Hand-write the provider block in `opencode.json` (project or global). Exact structure from
+   https://opencode.ai/docs/providers/ (fetched directly, JSON quoted verbatim from the docs):
+
+```json
+{
+  "$schema": "https://opencode.ai/config.json",
+  "provider": {
+    "llamacpp": {
+      "npm": "@ai-sdk/openai-compatible",
+      "name": "llama.cpp (local)",
+      "options": {
+        "baseURL": "http://localhost:8080/v1",
+        "apiKey": "sk-local-not-checked"
+      },
+      "models": {
+        "qwen3.8-27b": {
+          "name": "Qwen3.8-27B",
+          "limit": {
+            "context": 131072,
+            "output": 8192
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+Key fields:
+- **`npm`**: `"@ai-sdk/openai-compatible"` — the Vercel AI SDK adapter OpenCode uses for any server
+  speaking `/v1/chat/completions`. This is what makes it work against llama.cpp's OpenAI-compatible
+  endpoint (this repo's `/v1`), not the Anthropic shim (`/v1/messages` — see §4/§5 below for why the
+  OpenAI-compatible endpoint is the correct target, not the Anthropic shim, for OpenCode specifically).
+- **`options.baseURL`**: the server URL up to and including `/v1` — for this repo,
+  `http://localhost:8080/v1`.
+- **`options.apiKey`**: docs show it as *optional*, using `"{env:VAR_NAME}"` syntax to pull from an
+  env var in the general/hosted-provider example. For a local server that doesn't check the key,
+  supplying any non-empty string (or omitting it) both appear to work per the docs' own local-provider
+  example, which omits `apiKey` entirely for a bare local baseURL. Because the AI SDK's OpenAI-compatible
+  client sometimes still requires a non-empty `Authorization` header to be well-formed, the safer choice
+  is a dummy literal string (e.g. `"sk-local-not-checked"`) rather than omitting the field —
+  **this specific claim (dummy string vs. omit) is not directly confirmed by a docs quote either way**;
+  treat as a practical recommendation, not a documented requirement. **Confidence: medium** on the
+  omit-vs-dummy distinction; **high** on baseURL/npm/models structure itself.
+- **`models.<model-id>`**: the model ID key must match what the server exposes (check
+  `curl http://localhost:8080/v1/models`). `limit.context` / `limit.output` are optional token-limit
+  hints OpenCode uses for context-window bookkeeping — not enforced by the server itself.
+- Then select the model in OpenCode with `<providerId>/<modelId>`, e.g. `llamacpp/qwen3.8-27b`
+  (format documented on the troubleshooting page: "Models should be referenced like so:
+  `<providerId>/<modelId>`"). Source: https://opencode.ai/docs/troubleshooting/.
+
+Source for the whole block: https://opencode.ai/docs/providers/ (fetched directly).
+**Confidence: high** on the JSON shape; the exact llama.cpp `baseURL`/port is this repo's own
+docker-compose (`8080`), not an assumption.
+
+## 4. Documented compatibility notes: self-hosted OpenAI-compatible servers generally
+
+Two relevant, **currently open/closed-not-planned** primary-source issues in `anomalyco/opencode`:
+
+- **Issue #20669** — "Default agent is brittle against local OpenAI-compatible tool-call quirks."
+  Documents two concrete failure modes against local backends (llama.cpp, LM Studio) in OpenCode's
+  agent loop:
+  1. The `bash` tool throws a hard error ("expected string, received undefined") if the model emits a
+     valid `command` but omits the `description` field — some local models don't reliably fill optional
+     tool-call fields the way hosted models do.
+  2. Some local backends return `finish_reason: "tool_calls"` together with an **empty** `tool_calls: []`
+     array; OpenCode's loop doesn't treat this as a normal stop and can loop/hang instead.
+  **Resolution: closed as "not planned"** (maintainers declined to add compatibility shims for this),
+  referencing PR #26419. No documented workaround from the maintainers.
+  Source: https://github.com/anomalyco/opencode/issues/20669 (fetched directly).
+  **Confidence: high** on the issue content; the "not planned" resolution means this is a **live,
+  acknowledged, unfixed risk** for any local OpenAI-compatible backend, not specific to llama.cpp.
+
+- **Issue #1890** — "OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error
+  unless `--jinja`; with `--jinja`, template crashes (reject filter)." llama.cpp-specific:
+  - OpenCode unconditionally includes `tools` scaffolding in the request even for plain-chat agents
+    with no tools configured.
+  - Without llama.cpp's `--jinja` server flag: llama.cpp rejects the request outright
+    (`"tools param requires --jinja flag"`).
+  - With `--jinja`: llama.cpp's Jinja renderer crashes on some models' chat templates
+    (`"Value is not callable: null"`, a `reject` filter llama.cpp's Jinja implementation doesn't
+    support) — a 500 error.
+  - Tested against OpenCode v0.4.40/v0.4.41 and a Qwen3-30B model.
+  - Workarounds discussed: run llama.cpp with `--jinja` and a minimal template that ignores tools, or
+    proxy requests to strip `tools`/`tool_choice` before they reach llama.cpp.
+  Source: https://github.com/anomalyco/opencode/issues/1890 (fetched directly).
+  **Confidence: high** on content; **medium** on current relevance to this repo's exact llama.cpp/model
+  version since the issue is against an older OpenCode build and doesn't confirm current-build behavior.
+
+**Practical implication for this repo**: this repo's llama.cpp container should be launched with
+`--jinja` (check `docker-compose.yml` / entrypoint flags) or tool-calling via OpenCode will hard-fail
+with a 500 immediately — and even with `--jinja`, Qwen-family chat templates have a documented history
+of crashing llama.cpp's Jinja renderer (see also this repo's own
+`docs/research/qwen3.8-27b-tool-calling.md`, which independently found llama.cpp's tool-call
+parser/grammar for the Qwen3.5 lineage — which Qwen3.8-27B shares — to be broken in multiple
+still-open llama.cpp upstream issues, unrelated to OpenCode).
+
+## 5. Model-specific notes for Qwen3.8-27B vs. what OpenCode expects from cloud models
+
+- **Tool-calling reliability is the dominant risk, not context/thinking-mode handling.** This repo's
+  own prior research (`docs/research/qwen3.8-27b-tool-calling.md`) independently concluded, via
+  llama.cpp upstream issues, that Qwen3.8-27B's architecture lineage (Qwen3.5, hybrid Gated
+  DeltaNet/attention) has **documented, still-open llama.cpp tool-call parser bugs** — this is a
+  llama.cpp-side problem that affects both the Anthropic shim and the plain OpenAI-compatible endpoint
+  identically, since the Anthropic shim just re-uses llama.cpp's normal chat-completions tool-calling
+  pipeline (source: https://github.com/ggml-org/llama.cpp/pull/17570, per that same file). Combined with
+  OpenCode's own §4 issues, expect tool-calling from OpenCode against this specific model to be
+  **unreliable until upstream llama.cpp fixes land**, independent of OpenCode config correctness.
+- **Long context**: OpenCode's `limit.context`/`limit.output` fields (§3) are just bookkeeping hints for
+  OpenCode's own context-management UI/truncation logic — they don't change what the server actually
+  accepts. Set them to match whatever `--ctx-size` this repo's llama.cpp container is actually launched
+  with (check `docker-compose.yml`), not a value assumed from the model card, or OpenCode may
+  miscalculate when to compact/summarize the conversation.
+- **Thinking/reasoning-mode handling**: OpenCode's documented reasoning-effort config
+  (`options.reasoningEffort`, `options.thinking.budgetTokens` — https://opencode.ai/docs/models/,
+  fetched directly) is written for providers with a **native reasoning-effort API parameter** (OpenAI's
+  `reasoningEffort`, Anthropic's `thinking` block) — i.e., the hosted provider itself understands the
+  knob and returns reasoning separately from the answer. **The docs do not document any handling for
+  models that instead emit inline `<think>...</think>` tags in the raw completion text** (the pattern
+  Qwen3-family "hybrid thinking" models typically use when reasoning isn't a first-class API field).
+  For a llama.cpp-served Qwen3.8-27B, `reasoningEffort`/`thinking` config keys almost certainly have
+  **no effect** (llama.cpp's OpenAI-compatible endpoint doesn't expose or consume those keys) — thinking
+  mode for this model would need to be controlled at the llama.cpp/prompt level (e.g., a
+  `/no_think` marker or template-level toggle, if Qwen3.8-27B's chat template supports one), and
+  whether OpenCode strips or garbles `<think>` blocks in the response was **not found documented
+  anywhere in the primary sources checked**. **Confidence: low** on this specific sub-claim — flagged
+  as an open question, not a verified fact. Recommend testing empirically once tool-calling is unblocked.
+
+## Sources consulted (primary)
+
+- https://opencode.ai/docs/ — install methods, docs nav
+- https://opencode.ai/docs/config/ — config file location/format
+- https://opencode.ai/docs/providers/ — custom OpenAI-compatible provider JSON
+- https://opencode.ai/docs/models/ — reasoning effort / thinking config
+- https://opencode.ai/docs/troubleshooting/ — model ID reference format
+- https://github.com/anomalyco/opencode — repo README, install commands, license
+- https://github.com/anomalyco/opencode/issues/20669 — local-backend tool-call brittleness (closed, not planned)
+- https://github.com/anomalyco/opencode/issues/1890 — llama.cpp `--jinja` / template crash
+- https://github.com/anomalyco/opencode/issues/8390, #6841, #16440 — org rename (`sst` → `anomalyco`) fallout
+- https://github.com/ggml-org/llama.cpp/pull/17570 — Anthropic Messages shim reuses OpenAI tool-calling pipeline (also cited in this repo's `docs/research/qwen3.8-27b-tool-calling.md`)
+- This repo: `docs/research/qwen3.8-27b-tool-calling.md` — independent finding of open llama.cpp tool-call parser bugs for the Qwen3.5/Qwen3.8 lineage
+
+## Confidence summary
+
+| Claim | Confidence |
+|---|---|
+| Correct project = `anomalyco/opencode` (formerly `sst/opencode`) | High |
+| Install commands | High |
+| Config file location/format | High |
+| Provider JSON shape (npm/baseURL/models) | High |
+| apiKey omit-vs-dummy for no-auth servers | Medium (not directly confirmed) |
+| OpenCode local-backend tool-call brittleness (#20669) | High (content); this is a live unfixed issue |
+| llama.cpp `--jinja` requirement / template crash (#1890) | High (content); Medium (currency vs. latest builds) |
+| Qwen3.8-27B thinking-tag handling in OpenCode | Low — undocumented, flagged as open question |