LLM-Server/docs/research/opencode-cli-setup.md

OpenCode CLI: install + local llama.cpp provider config

Date: 2026-08-24 Scope: How to install OpenCode CLI and point it at this repo's llama.cpp OpenAI-compatible server (http://localhost:8080/v1, serving Qwen3.8-27B) instead of a hosted provider.

0. Which "opencode" is this?

Confirmed project: the OpenCode coding agent, docs at https://opencode.ai/docs/, repo currently at github.com/anomalyco/opencode.

• The repo was formerly sst/opencode and was moved to anomalyco/opencode as part of the SST team's org rebrand to "Anomaly." Multiple in-repo issues track fallout from the move (stale ghcr.io/sst/opencode Docker tips, Homebrew tap migration, tool integrations still pointing at the old path). Sources: https://github.com/anomalyco/opencode/issues/8390 , https://github.com/anomalyco/opencode/issues/6841 , https://github.com/anomalyco/opencode/issues/16440 , https://news.ycombinator.com/item?id=46552218 (community confirmation of the org move). Confidence: high — corroborated by several independent primary-source issues in the same repo, though I did not find an explicit "we renamed the org" announcement post to cite as the single authoritative statement (the docs site itself just says "OpenCode" with no history section).
• Naming collision to avoid: opencode-ai/opencode (a different, unrelated Go/Bubble Tea TUI project, MIT licensed, also calling itself "a powerful AI coding agent") turned up in search results and is not this project. The ticket's target — sst/opencode — is the TypeScript/Bun project now at anomalyco/opencode, confirmed by docs domain opencode.ai matching the install script URL used by both the old and current repo. Do not install opencode-ai/opencode by mistake — check opencode --version output / package name (opencode-ai on npm) if unsure. Source: https://github.com/opencode-ai/opencode (surfaced in search, cross-checked and excluded).

1. Install

Official methods, from https://opencode.ai/docs/ and https://github.com/anomalyco/opencode README (fetched directly):

# curl install script (recommended by the docs)
curl -fsSL https://opencode.ai/install | bash

# npm
npm i -g opencode-ai@latest

# Homebrew
brew install anomalyco/tap/opencode

Also documented as available: Bun, pnpm, Yarn, Arch pacman/paru, Windows Chocolatey/Scoop, Mise, Docker. Source: https://opencode.ai/docs/ (Installation section).

Confidence: high — these are the exact commands returned by fetching the docs and GitHub README directly, not a paraphrase from a third-party blog.

2. Config file: location and format

Format: JSON or JSONC (JSON with comments). File name: opencode.json (or opencode.jsonc).

• Global: ~/.config/opencode/opencode.json — user-wide settings.
• Project: opencode.json in the project root — picked up per-project, overrides/merges with global.
• A separate tui.json (same locations) holds TUI-only settings (themes, keybinds) — not needed for provider config.
• Schema for editor validation/autocomplete: "$schema": "https://opencode.ai/config.json".

Source: https://opencode.ai/docs/config/ (fetched directly). Confidence: high.

3. Declaring a local OpenAI-compatible provider (llama.cpp)

Two-part setup per the docs:

1. Optionally run /connect inside OpenCode, choose "Other", and give the provider a unique ID — this only stores a credential (which can be left blank/dummy for a local no-auth server) and does not generate the provider config block itself.
2. Hand-write the provider block in opencode.json (project or global). Exact structure from https://opencode.ai/docs/providers/ (fetched directly, JSON quoted verbatim from the docs):

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llamacpp": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "llama.cpp (local)",
      "options": {
        "baseURL": "http://localhost:8080/v1",
        "apiKey": "sk-local-not-checked"
      },
      "models": {
        "qwen3.8-27b": {
          "name": "Qwen3.8-27B",
          "limit": {
            "context": 131072,
            "output": 8192
          }
        }
      }
    }
  }
}

Key fields:

• npm: "@ai-sdk/openai-compatible" — the Vercel AI SDK adapter OpenCode uses for any server speaking /v1/chat/completions. This is what makes it work against llama.cpp's OpenAI-compatible endpoint (this repo's /v1), not the Anthropic shim (/v1/messages — see §4/§5 below for why the OpenAI-compatible endpoint is the correct target, not the Anthropic shim, for OpenCode specifically).
• options.baseURL: the server URL up to and including /v1 — for this repo, http://localhost:8080/v1.
• options.apiKey: docs show it as optional, using "{env:VAR_NAME}" syntax to pull from an env var in the general/hosted-provider example. For a local server that doesn't check the key, supplying any non-empty string (or omitting it) both appear to work per the docs' own local-provider example, which omits apiKey entirely for a bare local baseURL. Because the AI SDK's OpenAI-compatible client sometimes still requires a non-empty Authorization header to be well-formed, the safer choice is a dummy literal string (e.g. "sk-local-not-checked") rather than omitting the field — this specific claim (dummy string vs. omit) is not directly confirmed by a docs quote either way; treat as a practical recommendation, not a documented requirement. Confidence: medium on the omit-vs-dummy distinction; high on baseURL/npm/models structure itself.
• models.<model-id>: the model ID key must match what the server exposes (check curl http://localhost:8080/v1/models). limit.context / limit.output are optional token-limit hints OpenCode uses for context-window bookkeeping — not enforced by the server itself.
• Then select the model in OpenCode with <providerId>/<modelId>, e.g. llamacpp/qwen3.8-27b (format documented on the troubleshooting page: "Models should be referenced like so: <providerId>/<modelId>"). Source: https://opencode.ai/docs/troubleshooting/.

Source for the whole block: https://opencode.ai/docs/providers/ (fetched directly). Confidence: high on the JSON shape; the exact llama.cpp baseURL/port is this repo's own docker-compose (8080), not an assumption.

4. Documented compatibility notes: self-hosted OpenAI-compatible servers generally

Two relevant, currently open/closed-not-planned primary-source issues in anomalyco/opencode:

• Issue #20669 — "Default agent is brittle against local OpenAI-compatible tool-call quirks." Documents two concrete failure modes against local backends (llama.cpp, LM Studio) in OpenCode's agent loop:

  1. The bash tool throws a hard error ("expected string, received undefined") if the model emits a valid command but omits the description field — some local models don't reliably fill optional tool-call fields the way hosted models do.
  2. Some local backends return finish_reason: "tool_calls" together with an empty tool_calls: [] array; OpenCode's loop doesn't treat this as a normal stop and can loop/hang instead. Resolution: closed as "not planned" (maintainers declined to add compatibility shims for this), referencing PR #26419. No documented workaround from the maintainers. Source: https://github.com/anomalyco/opencode/issues/20669 (fetched directly). Confidence: high on the issue content; the "not planned" resolution means this is a live, acknowledged, unfixed risk for any local OpenAI-compatible backend, not specific to llama.cpp.

• Issue #1890 — "OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error unless --jinja; with --jinja, template crashes (reject filter)." llama.cpp-specific:

  • OpenCode unconditionally includes tools scaffolding in the request even for plain-chat agents with no tools configured.
  • Without llama.cpp's --jinja server flag: llama.cpp rejects the request outright ("tools param requires --jinja flag").
  • With --jinja: llama.cpp's Jinja renderer crashes on some models' chat templates ("Value is not callable: null", a reject filter llama.cpp's Jinja implementation doesn't support) — a 500 error.
  • Tested against OpenCode v0.4.40/v0.4.41 and a Qwen3-30B model.
  • Workarounds discussed: run llama.cpp with --jinja and a minimal template that ignores tools, or proxy requests to strip tools/tool_choice before they reach llama.cpp. Source: https://github.com/anomalyco/opencode/issues/1890 (fetched directly). Confidence: high on content; medium on current relevance to this repo's exact llama.cpp/model version since the issue is against an older OpenCode build and doesn't confirm current-build behavior.

Practical implication for this repo: this repo's llama.cpp container should be launched with --jinja (check docker-compose.yml / entrypoint flags) or tool-calling via OpenCode will hard-fail with a 500 immediately — and even with --jinja, Qwen-family chat templates have a documented history of crashing llama.cpp's Jinja renderer (see also this repo's own docs/research/qwen3.8-27b-tool-calling.md, which independently found llama.cpp's tool-call parser/grammar for the Qwen3.5 lineage — which Qwen3.8-27B shares — to be broken in multiple still-open llama.cpp upstream issues, unrelated to OpenCode).

5. Model-specific notes for Qwen3.8-27B vs. what OpenCode expects from cloud models

• Tool-calling reliability is the dominant risk, not context/thinking-mode handling. This repo's own prior research (docs/research/qwen3.8-27b-tool-calling.md) independently concluded, via llama.cpp upstream issues, that Qwen3.8-27B's architecture lineage (Qwen3.5, hybrid Gated DeltaNet/attention) has documented, still-open llama.cpp tool-call parser bugs — this is a llama.cpp-side problem that affects both the Anthropic shim and the plain OpenAI-compatible endpoint identically, since the Anthropic shim just re-uses llama.cpp's normal chat-completions tool-calling pipeline (source: https://github.com/ggml-org/llama.cpp/pull/17570, per that same file). Combined with OpenCode's own §4 issues, expect tool-calling from OpenCode against this specific model to be unreliable until upstream llama.cpp fixes land, independent of OpenCode config correctness.
• Long context: OpenCode's limit.context/limit.output fields (§3) are just bookkeeping hints for OpenCode's own context-management UI/truncation logic — they don't change what the server actually accepts. Set them to match whatever --ctx-size this repo's llama.cpp container is actually launched with (check docker-compose.yml), not a value assumed from the model card, or OpenCode may miscalculate when to compact/summarize the conversation.
• Thinking/reasoning-mode handling: OpenCode's documented reasoning-effort config (options.reasoningEffort, options.thinking.budgetTokens — https://opencode.ai/docs/models/, fetched directly) is written for providers with a native reasoning-effort API parameter (OpenAI's reasoningEffort, Anthropic's thinking block) — i.e., the hosted provider itself understands the knob and returns reasoning separately from the answer. The docs do not document any handling for models that instead emit inline <think>...</think> tags in the raw completion text (the pattern Qwen3-family "hybrid thinking" models typically use when reasoning isn't a first-class API field). For a llama.cpp-served Qwen3.8-27B, reasoningEffort/thinking config keys almost certainly have no effect (llama.cpp's OpenAI-compatible endpoint doesn't expose or consume those keys) — thinking mode for this model would need to be controlled at the llama.cpp/prompt level (e.g., a /no_think marker or template-level toggle, if Qwen3.8-27B's chat template supports one), and whether OpenCode strips or garbles <think> blocks in the response was not found documented anywhere in the primary sources checked. Confidence: low on this specific sub-claim — flagged as an open question, not a verified fact. Recommend testing empirically once tool-calling is unblocked.

Sources consulted (primary)

• https://opencode.ai/docs/ — install methods, docs nav
• https://opencode.ai/docs/config/ — config file location/format
• https://opencode.ai/docs/providers/ — custom OpenAI-compatible provider JSON
• https://opencode.ai/docs/models/ — reasoning effort / thinking config
• https://opencode.ai/docs/troubleshooting/ — model ID reference format
• https://github.com/anomalyco/opencode — repo README, install commands, license
• https://github.com/anomalyco/opencode/issues/20669 — local-backend tool-call brittleness (closed, not planned)
• https://github.com/anomalyco/opencode/issues/1890 — llama.cpp --jinja / template crash
• https://github.com/anomalyco/opencode/issues/8390, #6841, #16440 — org rename (sst → anomalyco) fallout
• https://github.com/ggml-org/llama.cpp/pull/17570 — Anthropic Messages shim reuses OpenAI tool-calling pipeline (also cited in this repo's docs/research/qwen3.8-27b-tool-calling.md)
• This repo: docs/research/qwen3.8-27b-tool-calling.md — independent finding of open llama.cpp tool-call parser bugs for the Qwen3.5/Qwen3.8 lineage

Confidence summary

Claim	Confidence
Correct project = anomalyco/opencode (formerly sst/opencode)	High
Install commands	High
Config file location/format	High
Provider JSON shape (npm/baseURL/models)	High
apiKey omit-vs-dummy for no-auth servers	Medium (not directly confirmed)
OpenCode local-backend tool-call brittleness (#20669)	High (content); this is a live unfixed issue
llama.cpp --jinja requirement / template crash (#1890)	High (content); Medium (currency vs. latest builds)
Qwen3.8-27B thinking-tag handling in OpenCode	Low — undocumented, flagged as open question