LLM-Server/docs/research/opencode-cli-setup.md

# OpenCode CLI: install + local llama.cpp provider config

**Date:** 2026-08-24
**Scope:** How to install OpenCode CLI and point it at this repo's llama.cpp OpenAI-compatible server
(`http://localhost:8080/v1`, serving Qwen3.8-27B) instead of a hosted provider.

## 0. Which "opencode" is this?

Confirmed project: **the OpenCode coding agent**, docs at https://opencode.ai/docs/, repo currently at
**github.com/anomalyco/opencode**.

- The repo was **formerly `sst/opencode`** and was moved to `anomalyco/opencode` as part of the SST
  team's org rebrand to "Anomaly." Multiple in-repo issues track fallout from the move (stale
  `ghcr.io/sst/opencode` Docker tips, Homebrew tap migration, tool integrations still pointing at the
  old path). Sources:
  https://github.com/anomalyco/opencode/issues/8390 ,
  https://github.com/anomalyco/opencode/issues/6841 ,
  https://github.com/anomalyco/opencode/issues/16440 ,
  https://news.ycombinator.com/item?id=46552218 (community confirmation of the org move).
  **Confidence: high** — corroborated by several independent primary-source issues in the same repo,
  though I did not find an explicit "we renamed the org" announcement post to cite as the single
  authoritative statement (the docs site itself just says "OpenCode" with no history section).
- **Naming collision to avoid**: `opencode-ai/opencode` (a different, unrelated Go/Bubble Tea TUI
  project, MIT licensed, also calling itself "a powerful AI coding agent") turned up in search results
  and is **not** this project. The ticket's target — `sst/opencode` — is the TypeScript/Bun project now
  at `anomalyco/opencode`, confirmed by docs domain `opencode.ai` matching the install script URL used
  by both the old and current repo. Do not install `opencode-ai/opencode` by mistake — check `opencode
  --version` output / package name (`opencode-ai` on npm) if unsure.
  Source: https://github.com/opencode-ai/opencode (surfaced in search, cross-checked and excluded).

## 1. Install

Official methods, from https://opencode.ai/docs/ and https://github.com/anomalyco/opencode README
(fetched directly):

```bash
# curl install script (recommended by the docs)
curl -fsSL https://opencode.ai/install | bash

# npm
npm i -g opencode-ai@latest

# Homebrew
brew install anomalyco/tap/opencode
```

Also documented as available: Bun, pnpm, Yarn, Arch `pacman`/`paru`, Windows Chocolatey/Scoop, Mise,
Docker. Source: https://opencode.ai/docs/ (Installation section).

**Confidence: high** — these are the exact commands returned by fetching the docs and GitHub README
directly, not a paraphrase from a third-party blog.

## 2. Config file: location and format

Format: **JSON or JSONC** (JSON with comments). File name: `opencode.json` (or `opencode.jsonc`).

- **Global**: `~/.config/opencode/opencode.json` — user-wide settings.
- **Project**: `opencode.json` in the project root — picked up per-project, overrides/merges with
  global.
- A separate `tui.json` (same locations) holds TUI-only settings (themes, keybinds) — not needed for
  provider config.
- Schema for editor validation/autocomplete: `"$schema": "https://opencode.ai/config.json"`.

Source: https://opencode.ai/docs/config/ (fetched directly).
**Confidence: high.**

## 3. Declaring a local OpenAI-compatible provider (llama.cpp)

Two-part setup per the docs:

1. Optionally run `/connect` inside OpenCode, choose "Other", and give the provider a unique ID — this
   only stores a credential (which can be left blank/dummy for a local no-auth server) and does **not**
   generate the provider config block itself.
2. Hand-write the provider block in `opencode.json` (project or global). Exact structure from
   https://opencode.ai/docs/providers/ (fetched directly, JSON quoted verbatim from the docs):

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llamacpp": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "llama.cpp (local)",
      "options": {
        "baseURL": "http://localhost:8080/v1",
        "apiKey": "sk-local-not-checked"
      },
      "models": {
        "qwen3.8-27b": {
          "name": "Qwen3.8-27B",
          "limit": {
            "context": 131072,
            "output": 8192
          }
        }
      }
    }
  }
}
```

Key fields:
- **`npm`**: `"@ai-sdk/openai-compatible"` — the Vercel AI SDK adapter OpenCode uses for any server
  speaking `/v1/chat/completions`. This is what makes it work against llama.cpp's OpenAI-compatible
  endpoint (this repo's `/v1`), not the Anthropic shim (`/v1/messages` — see §4/§5 below for why the
  OpenAI-compatible endpoint is the correct target, not the Anthropic shim, for OpenCode specifically).
- **`options.baseURL`**: the server URL up to and including `/v1` — for this repo,
  `http://localhost:8080/v1`.
- **`options.apiKey`**: docs show it as *optional*, using `"{env:VAR_NAME}"` syntax to pull from an
  env var in the general/hosted-provider example. For a local server that doesn't check the key,
  supplying any non-empty string (or omitting it) both appear to work per the docs' own local-provider
  example, which omits `apiKey` entirely for a bare local baseURL. Because the AI SDK's OpenAI-compatible
  client sometimes still requires a non-empty `Authorization` header to be well-formed, the safer choice
  is a dummy literal string (e.g. `"sk-local-not-checked"`) rather than omitting the field —
  **this specific claim (dummy string vs. omit) is not directly confirmed by a docs quote either way**;
  treat as a practical recommendation, not a documented requirement. **Confidence: medium** on the
  omit-vs-dummy distinction; **high** on baseURL/npm/models structure itself.
- **`models.<model-id>`**: the model ID key must match what the server exposes (check
  `curl http://localhost:8080/v1/models`). `limit.context` / `limit.output` are optional token-limit
  hints OpenCode uses for context-window bookkeeping — not enforced by the server itself.
- Then select the model in OpenCode with `<providerId>/<modelId>`, e.g. `llamacpp/qwen3.8-27b`
  (format documented on the troubleshooting page: "Models should be referenced like so:
  `<providerId>/<modelId>`"). Source: https://opencode.ai/docs/troubleshooting/.

Source for the whole block: https://opencode.ai/docs/providers/ (fetched directly).
**Confidence: high** on the JSON shape; the exact llama.cpp `baseURL`/port is this repo's own
docker-compose (`8080`), not an assumption.

## 4. Documented compatibility notes: self-hosted OpenAI-compatible servers generally

Two relevant, **currently open/closed-not-planned** primary-source issues in `anomalyco/opencode`:

- **Issue #20669** — "Default agent is brittle against local OpenAI-compatible tool-call quirks."
  Documents two concrete failure modes against local backends (llama.cpp, LM Studio) in OpenCode's
  agent loop:
  1. The `bash` tool throws a hard error ("expected string, received undefined") if the model emits a
     valid `command` but omits the `description` field — some local models don't reliably fill optional
     tool-call fields the way hosted models do.
  2. Some local backends return `finish_reason: "tool_calls"` together with an **empty** `tool_calls: []`
     array; OpenCode's loop doesn't treat this as a normal stop and can loop/hang instead.
  **Resolution: closed as "not planned"** (maintainers declined to add compatibility shims for this),
  referencing PR #26419. No documented workaround from the maintainers.
  Source: https://github.com/anomalyco/opencode/issues/20669 (fetched directly).
  **Confidence: high** on the issue content; the "not planned" resolution means this is a **live,
  acknowledged, unfixed risk** for any local OpenAI-compatible backend, not specific to llama.cpp.

- **Issue #1890** — "OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error
  unless `--jinja`; with `--jinja`, template crashes (reject filter)." llama.cpp-specific:
  - OpenCode unconditionally includes `tools` scaffolding in the request even for plain-chat agents
    with no tools configured.
  - Without llama.cpp's `--jinja` server flag: llama.cpp rejects the request outright
    (`"tools param requires --jinja flag"`).
  - With `--jinja`: llama.cpp's Jinja renderer crashes on some models' chat templates
    (`"Value is not callable: null"`, a `reject` filter llama.cpp's Jinja implementation doesn't
    support) — a 500 error.
  - Tested against OpenCode v0.4.40/v0.4.41 and a Qwen3-30B model.
  - Workarounds discussed: run llama.cpp with `--jinja` and a minimal template that ignores tools, or
    proxy requests to strip `tools`/`tool_choice` before they reach llama.cpp.
  Source: https://github.com/anomalyco/opencode/issues/1890 (fetched directly).
  **Confidence: high** on content; **medium** on current relevance to this repo's exact llama.cpp/model
  version since the issue is against an older OpenCode build and doesn't confirm current-build behavior.

**Practical implication for this repo**: this repo's llama.cpp container should be launched with
`--jinja` (check `docker-compose.yml` / entrypoint flags) or tool-calling via OpenCode will hard-fail
with a 500 immediately — and even with `--jinja`, Qwen-family chat templates have a documented history
of crashing llama.cpp's Jinja renderer (see also this repo's own
`docs/research/qwen3.8-27b-tool-calling.md`, which independently found llama.cpp's tool-call
parser/grammar for the Qwen3.5 lineage — which Qwen3.8-27B shares — to be broken in multiple
still-open llama.cpp upstream issues, unrelated to OpenCode).

## 5. Model-specific notes for Qwen3.8-27B vs. what OpenCode expects from cloud models

- **Tool-calling reliability is the dominant risk, not context/thinking-mode handling.** This repo's
  own prior research (`docs/research/qwen3.8-27b-tool-calling.md`) independently concluded, via
  llama.cpp upstream issues, that Qwen3.8-27B's architecture lineage (Qwen3.5, hybrid Gated
  DeltaNet/attention) has **documented, still-open llama.cpp tool-call parser bugs** — this is a
  llama.cpp-side problem that affects both the Anthropic shim and the plain OpenAI-compatible endpoint
  identically, since the Anthropic shim just re-uses llama.cpp's normal chat-completions tool-calling
  pipeline (source: https://github.com/ggml-org/llama.cpp/pull/17570, per that same file). Combined with
  OpenCode's own §4 issues, expect tool-calling from OpenCode against this specific model to be
  **unreliable until upstream llama.cpp fixes land**, independent of OpenCode config correctness.
- **Long context**: OpenCode's `limit.context`/`limit.output` fields (§3) are just bookkeeping hints for
  OpenCode's own context-management UI/truncation logic — they don't change what the server actually
  accepts. Set them to match whatever `--ctx-size` this repo's llama.cpp container is actually launched
  with (check `docker-compose.yml`), not a value assumed from the model card, or OpenCode may
  miscalculate when to compact/summarize the conversation.
- **Thinking/reasoning-mode handling**: OpenCode's documented reasoning-effort config
  (`options.reasoningEffort`, `options.thinking.budgetTokens` — https://opencode.ai/docs/models/,
  fetched directly) is written for providers with a **native reasoning-effort API parameter** (OpenAI's
  `reasoningEffort`, Anthropic's `thinking` block) — i.e., the hosted provider itself understands the
  knob and returns reasoning separately from the answer. **The docs do not document any handling for
  models that instead emit inline `<think>...</think>` tags in the raw completion text** (the pattern
  Qwen3-family "hybrid thinking" models typically use when reasoning isn't a first-class API field).
  For a llama.cpp-served Qwen3.8-27B, `reasoningEffort`/`thinking` config keys almost certainly have
  **no effect** (llama.cpp's OpenAI-compatible endpoint doesn't expose or consume those keys) — thinking
  mode for this model would need to be controlled at the llama.cpp/prompt level (e.g., a
  `/no_think` marker or template-level toggle, if Qwen3.8-27B's chat template supports one), and
  whether OpenCode strips or garbles `<think>` blocks in the response was **not found documented
  anywhere in the primary sources checked**. **Confidence: low** on this specific sub-claim — flagged
  as an open question, not a verified fact. Recommend testing empirically once tool-calling is unblocked.

## Sources consulted (primary)

- https://opencode.ai/docs/ — install methods, docs nav
- https://opencode.ai/docs/config/ — config file location/format
- https://opencode.ai/docs/providers/ — custom OpenAI-compatible provider JSON
- https://opencode.ai/docs/models/ — reasoning effort / thinking config
- https://opencode.ai/docs/troubleshooting/ — model ID reference format
- https://github.com/anomalyco/opencode — repo README, install commands, license
- https://github.com/anomalyco/opencode/issues/20669 — local-backend tool-call brittleness (closed, not planned)
- https://github.com/anomalyco/opencode/issues/1890 — llama.cpp `--jinja` / template crash
- https://github.com/anomalyco/opencode/issues/8390, #6841, #16440 — org rename (`sst` → `anomalyco`) fallout
- https://github.com/ggml-org/llama.cpp/pull/17570 — Anthropic Messages shim reuses OpenAI tool-calling pipeline (also cited in this repo's `docs/research/qwen3.8-27b-tool-calling.md`)
- This repo: `docs/research/qwen3.8-27b-tool-calling.md` — independent finding of open llama.cpp tool-call parser bugs for the Qwen3.5/Qwen3.8 lineage

## Confidence summary

| Claim | Confidence |
|---|---|
| Correct project = `anomalyco/opencode` (formerly `sst/opencode`) | High |
| Install commands | High |
| Config file location/format | High |
| Provider JSON shape (npm/baseURL/models) | High |
| apiKey omit-vs-dummy for no-auth servers | Medium (not directly confirmed) |
| OpenCode local-backend tool-call brittleness (#20669) | High (content); this is a live unfixed issue |
| llama.cpp `--jinja` requirement / template crash (#1890) | High (content); Medium (currency vs. latest builds) |
| Qwen3.8-27B thinking-tag handling in OpenCode | Low — undocumented, flagged as open question |