LLM-Server/docs/research/opencode-auto-compact.md

OpenCode CLI: how auto-compact actually decides to trigger

Date: 2026-09-03 Scope: Resolves haylan/LLM-Server issue #27 (part of #26) — the config key(s), trigger threshold/formula, reasoning-token accounting, and per-model tunability of OpenCode's context-compaction behavior, and whether any of it is specific to hosted providers vs. the @ai-sdk/openai-compatible path this repo's docs/research/opencode-cli-setup.md documents for llama.cpp/litellm-style backends.

Freshness note: opencode-cli-setup.md (dated 2026-08-24) does not cover compaction at all beyond one sentence ("§5: OpenCode's limit.context/limit.output fields ... don't change what the server actually accepts ... may miscalculate when to compact/summarize"). This document supersedes that gap. It is based on:

1. Live docs fetches from https://opencode.ai/docs/ (2026-09-03), and
2. A fresh git clone of https://github.com/anomalyco/opencode at commit b578b7261fc9ec4917fe272df5cc4bd8a056cd5d (2026-09-03T09:47:21+08:00 — same day as this research), package.json version 1.18.27. Source-code claims below are cited by file path in that clone and are the highest-confidence source in this doc — they're what actually ships, not a doc description or a third-party claim.

Confidence scheme

• High — read directly from the current source code in the repo, or a docs page quoted verbatim.
• Medium — inferred from source code behavior that isn't spelled out in a single line/comment (i.e., I traced call sites to confirm it, rather than reading one authoritative line).
• Low — plausible but not directly confirmed in the sources checked; flagged as an open question.

---

Verdict summary (answers to the four questions asked in #27)

Question	Answer	Confidence
Config key(s) controlling threshold/behavior	Single global top-level compaction object in opencode.json: auto, prune, reserved, tail_turns, preserve_recent_tokens. No threshold percentage key exists.	High
Is it a hardcoded percentage of limit.context?	No. It's usedTokens >= (contextLimit − reservedBuffer), where reservedBuffer defaults to min(20_000, min(model.limit.output, 32_768) or default), i.e. compaction reserves room for one more max-size reply, not a flat 75%/95% cutoff. Some closed GitHub feature requests describe it as "hardcoded 75%" — that claim is not what the current source does (see §2).	High (source), contradicts a stale community claim (see §2)
Does compaction count reasoning/reasoning_content tokens?	Yes, in practice, via the provider's usage.total_tokens (which for llama.cpp/litellm includes every generated token, reasoning or not) — but the token bookkeeping OpenCode itself derives (tokens.output, tokens.reasoning) explicitly splits reasoning out of output, and the compaction trigger's own fallback arithmetic (used only if the provider omits total_tokens) omits tokens.reasoning entirely. See §3 for the exact mechanism and the one edge case where reasoning tokens could be undercounted.	High (source), Medium (edge-case behavior when total_tokens is absent)
Per-model or single global behavior?	Global only. The compaction block is a top-level config key, not nested under provider.<id>.models.<id> or any per-model schema. It cannot be disabled or tuned for one model while enabled for another. The indirect lever is each model's own limit.context/limit.output (which you already set per-model for custom providers), since those numbers feed the same global formula per-model. Multiple GitHub feature requests (#11314, #11930, #8140, #16375) ask for per-model/per-agent configurability; all are open or closed-not-planned as of this check.	High

---

1. The config surface (verbatim from source + docs)

Global (or project) opencode.json:

{
  "compaction": {
    "auto": true,
    "prune": false,
    "reserved": 20000,
    "tail_turns": null,
    "preserve_recent_tokens": null
  }
}

Field descriptions, quoted verbatim from the config schema (packages/core/src/v1/config/config.ts, lines ~149–166 in the cloned repo):

• auto — "Enable automatic compaction when context is full (default: true)"
• prune — "Enable pruning of old tool outputs (default: false)"
• tail_turns — "Maximum number of recent user turns, including their following assistant/tool responses, to keep verbatim during compaction. By default retention is limited only by the preserved token budget."
• preserve_recent_tokens — "Maximum number of tokens from recent turns to preserve verbatim after compaction"
• reserved — "Token buffer for compaction. Leaves enough window to avoid overflow during compaction."

Sources:

• https://opencode.ai/docs/config/ (fetched 2026-09-03) confirms auto/prune/reserved with the same defaults and descriptions; the docs page does not mention tail_turns or preserve_recent_tokens — those two are documented only in the source schema, not (yet) on the public docs page. Confidence: High on all five keys existing and their defaults; the docs-vs-source gap on the last two is itself notable (docs page is behind the schema).
• packages/core/src/v1/config/config.ts (cloned repo, ~line 149) — schema + descriptions.

There is no threshold, percent, or similarly-named key anywhere in the config schema. I grepped packages/core/src/v1/config/ and packages/core/src/config.ts for compaction and found only the struct above — no percentage field exists in the current schema (High confidence; direct grep of current source).

Not nested under provider/model. The compaction key sits at the top level of opencode.json, a sibling of provider, model, agent, etc. — not inside provider.<id>.models.<model-id> (the block this repo's opencode-cli-setup.md §3 documents for declaring the llama.cpp provider). Confirmed by reading the full top-level config struct in packages/core/src/v1/config/config.ts (~lines 95–170): compaction is a direct sibling of provider, not a child of it.

There is a separate, easily-confused concept: agent.compaction (also in that same file, ~line 105) — this only lets you assign a different agent/model to perform the summarization step itself (e.g. run the compaction LLM call on a cheaper model), not a per-model threshold override. It does not change when compaction triggers.

2. The actual trigger formula (not a flat percentage)

Live code, packages/opencode/src/session/overflow.ts (the shipped/default compaction-trigger path — see "which code path ships" note at the end of this section):

const COMPACTION_BUFFER = 20_000

export function usable(input: { cfg: ConfigV1.Info; model: Provider.Model; outputTokenMax?: number }) {
  const context = input.model.limit.context
  if (context === 0) return 0

  const reserved =
    input.cfg.compaction?.reserved ??
    Math.min(COMPACTION_BUFFER, ProviderTransform.maxOutputTokens(input.model, input.outputTokenMax))
  return input.model.limit.input
    ? Math.max(0, input.model.limit.input - reserved)
    : Math.max(0, context - ProviderTransform.maxOutputTokens(input.model, input.outputTokenMax))
}

export function isOverflow(input: {
  cfg: ConfigV1.Info
  tokens: SessionV1.Assistant["tokens"]
  model: Provider.Model
  outputTokenMax?: number
}) {
  if (input.cfg.compaction?.auto === false) return false
  if (input.model.limit.context === 0) return false

  const count =
    input.tokens.total || input.tokens.input + input.tokens.output + input.tokens.cache.read + input.tokens.cache.write
  return count >= usable(input)
}

In plain terms:

• Compaction triggers when the running token count (count) reaches or exceeds usable = contextLimit − reservedBuffer (or, if the model declares a separate limit.input, usable = limit.input − reservedBuffer instead).
• reservedBuffer defaults to min(20_000, maxOutputTokens), where maxOutputTokens = min(model.limit.output, 32_768) || 32_768 (OUTPUT_TOKEN_MAX constant, packages/opencode/src/provider/transform.ts — maxOutputTokens() at ~line 1468). It can be overridden with compaction.reserved.
• If model.limit.context is 0 or unset, compaction is silently disabled entirely — this matters for custom @ai-sdk/openai-compatible providers where limit.context is a value the user must supply by hand (per opencode-cli-setup.md §3); omit it, and auto-compact never fires for that model.
• Compaction is disabled outright if compaction.auto === false.

This is not a fixed 75%/90%/95%-of-context cutoff. It's context − reservedOutputBuffer, which in practice usually lands somewhere in the 80–95%+ range depending on the model's own limit.output relative to limit.context — but it's a token-count subtraction, not a percentage multiplication, and there is no config key to set a percentage.

On the "hardcoded 75%" claim: a closed GitHub feature request (https://github.com/anomalyco/opencode/issues/11314, "Feature Request: Configurable Context Compaction Threshold") asserts "Currently, OpenCode triggers context compaction at a hardcoded 75% threshold of a model's context window." That is a user's claim in a feature-request issue, not a maintainer statement or a source citation, and it does not match the formula actually in the current source (which is a token-count subtraction tied to output-buffer size, not a flat percentage). Treat that issue as evidence that users perceive/want configurability, not as an accurate description of the mechanism. A second, similarly-shaped request (https://github.com/anomalyco/opencode/issues/11930) instead describes the current default as "100% threshold" — the two community reports disagree with each other, which is itself a signal neither is a reliable description of internals; the source code is what should be trusted here. Confidence: High on the source-code formula; High that the 75%/100% claims in those two issues are user speculation, not verified facts — both are closed as "not planned" with no maintainer confirmation of the underlying mechanism in the content fetched.

Which code path ships: the repo contains two parallel implementations — the one quoted above (packages/opencode/src/session/overflow.ts + packages/opencode/src/session/compaction.ts, using ConfigV1/SessionV1 types) and a newer, structurally different one (packages/core/src/session/compaction.ts + packages/core/src/session/runner/llm.ts, using Config.Entry/SessionMessage types, with its own compactIfNeeded/compactAfterOverflow functions and a buffer config field instead of reserved). Tracing the call path: the newer runner is only reached when flags.experimentalNativeLlm is true, gated by the OPENCODE_EXPERIMENTAL_NATIVE_LLM env var (packages/opencode/src/effect/runtime-flags.ts, packages/opencode/src/session/llm.ts line ~226) — off by default. The overflow.ts/V1 path quoted above is what ships by default in v1.18.27. If this repo's OpenCode setup ever sets OPENCODE_EXPERIMENTAL_NATIVE_LLM=1 (or a future release flips the default), the newer engine's compactIfNeeded() uses a structurally similar but not identical check: estimatedPromptTokens <= context − max(output, config.buffer) — same shape (context minus a reserved buffer, no percentage), different field name (buffer vs reserved). Confidence: High on which path ships by default; Medium on exact behavior differences if the experimental flag is ever turned on, since I did not exhaustively diff every line of the newer engine.

3. Reasoning/thinking-token accounting — the Qwen3.8 angle

This repo's litellm-config.yaml flags that Qwen3.8-27B (qwen3.8-27b-local) spends generation tokens on reasoning_content before content:

# Qwen3 is a reasoning model — it spends output tokens on
# reasoning_content before ever writing content. ...
max_tokens: 16384

(/home/haylan/Projects/LLM-Server/litellm-config.yaml, qwen3.8-27b-local model block.)

OpenCode's own token bookkeeping (packages/opencode/src/session/session.ts, ~lines 340–375, the function that turns a provider's raw usage object into the tokens struct stored on each assistant message) does this:

const inputTokens = safe(input.usage.inputTokens ?? 0)
const outputTokens = safe(input.usage.outputTokens ?? 0)
const reasoningTokens = safe(input.usage.reasoningTokens ?? 0)
...
const total = input.usage.totalTokens

const tokens = {
  total,
  input: adjustedInputTokens,
  output: safe(outputTokens - reasoningTokens),   // reasoning is subtracted OUT of "output"
  reasoning: reasoningTokens,                      // tracked as its own field
  cache: { write: cacheWriteInputTokens, read: cacheReadInputTokens },
}

And the compaction trigger (overflow.ts, quoted in §2) computes:

const count = input.tokens.total || input.tokens.input + input.tokens.output + input.tokens.cache.read + input.tokens.cache.write

Two things follow:

1. tokens.reasoning is never added back in by the fallback sum (input + output + cache.read + cache.write) — that expression has no + reasoning term. If the compaction trigger ever fell back to this sum (i.e., the provider didn't return totalTokens), reasoning tokens spent on reasoning_content would be excluded from the overflow calculation, undercounting real context usage.
2. In the normal case, total is used instead of the fallback sum, and total = usage.totalTokens straight from the provider's raw response — computed by the provider/AI-SDK before OpenCode splits outputTokens into output/reasoning. Since the AI SDK's OpenAI-compatible adapter (and litellm/llama.cpp underneath it) counts every generated token — reasoning and content alike — as part of completion_tokens/total_tokens, total does include reasoning tokens in this normal path. So in practice, for a llama.cpp/litellm backend that reports usage.total_tokens on every response (the OpenAI chat-completions spec requires this field), reasoning tokens are accounted for in the compaction trigger via total, not via the explicit reasoning field.

The edge case that would matter for this repo: if litellm or llama.cpp's OpenAI-compatible endpoint ever omitted usage.total_tokens from a response (malformed/incomplete usage block — this has happened with some llama.cpp server versions/flags), OpenCode's fallback sum would silently undercount by the full reasoning amount, delaying compaction past the point it should have triggered and increasing risk of a hard context_length_exceeded — exactly the failure mode reported by an unrelated user in https://github.com/anomalyco/opencode/issues/8089 ("Auto-compaction enabled by default, but context_length_exceeded errors still occur in agent workflows"), though that issue's cause was not confirmed to be this specific gap (it involved OpenAI's GPT-5.2 and multi-agent/subagent workflows, not a local llama.cpp backend, and the issue thread contains no maintainer diagnosis of root cause in the content fetched).

Confidence: High on the source-code mechanics described (the output = outputTokens − reasoningTokens split, the total || sum fallback, and the missing + reasoning term in the fallback). Medium on whether this repo's specific llama.cpp/litellm stack reliably returns usage.total_tokens on every response for the qwen3.8-27b-local model — this was not verified against a live request/response in this research pass (would need an empirical check: hit http://localhost:8080/v1/chat/completions directly or via the litellm proxy and inspect the usage block of an actual reasoning response). Recommend that empirical check as a fast follow if this matters operationally.

4. Per-model tunability

Confirmed absent, both from the schema (§1) and from community feature requests asking for exactly this and not getting it:

• https://github.com/anomalyco/opencode/issues/11314 — "Feature Request: Configurable Context Compaction Threshold" — requests a compaction.threshold with "optional per-model overrides." Closed as not planned (per WebFetch of the issue).
• https://github.com/anomalyco/opencode/issues/11930 — "Feature: Configurable compaction threshold and model (global + per-model)" — explicitly requests global and per-model threshold config. Closed as not planned, no maintainer reply visible in the content fetched.
• https://github.com/anomalyco/opencode/issues/8140 — "Feature Request: Configurable context limit and auto-compaction threshold" — same theme (title only confirmed via search; not individually fetched in this pass).
• https://github.com/anomalyco/opencode/issues/16375 — "[FEATURE]: Per-agent compaction config (disable compaction for specific agents)" — same theme, per-agent instead of per-model (title only confirmed via search; not individually fetched in this pass).

All four are open/closed-not-planned as of 2026-09-03 — i.e., as of this check, none of this has shipped: compaction remains a single global on/off + buffer-size knob, with no per-model or per-agent threshold override. Confidence: High that the feature doesn't exist in the schema (direct source read); Medium on the exact current status of #8140/#16375 specifically since only their titles were confirmed via search results, not their full issue bodies.

The one indirect per-model lever that does exist: since usable() (§2) reads input.model.limit.context / input.model.limit.input / input.model.limit.output — all per-model fields already documented in opencode-cli-setup.md §3/§5 for custom providers — setting those numbers differently per model in the provider.<id>.models.<model-id>.limit block changes where that model's compaction fires, without needing a dedicated per-model compaction key. This is bookkeeping-hint tuning, not a first-class "compaction threshold" feature.

5. Is any of this specific to hosted/built-in providers vs. @ai-sdk/openai-compatible?

No. The entire trigger path (overflow.ts, compaction.ts) operates only on Provider.Model (a normalized model descriptor with limit.context/limit.input/limit.output) and the message tokens struct built from the SDK's generic usage object (session.ts, §3) — nothing in the compaction code branches on model.api.npm or provider identity. The mechanism is provider-agnostic by construction: any provider adapter that populates usage (inputTokens/outputTokens/totalTokens) and any model entry that has a nonzero limit.context gets the same compaction behavior, including a hand-declared @ai-sdk/openai-compatible provider block like this repo's llamacpp provider in opencode-cli-setup.md §3. Confidence: High — read directly from the trigger/accounting source, which takes no provider-specific branch.

The one place this repo needs to be careful about, restated from §2: for a custom @ai-sdk/openai-compatible provider, limit.context (and ideally limit.output) must be set by hand in opencode.json to match the real --ctx-size the llama.cpp container is launched with — if left unset (limit.context defaults to 0 for an unrecognized custom model), compaction is silently disabled for that model rather than silently misfiring.

Sources consulted (primary)

• https://opencode.ai/docs/ — nav/sitemap fetch (2026-09-03); confirms no dedicated "context management"/"compaction" page exists in the current docs nav — it lives only in the Config reference.
• https://opencode.ai/docs/config/ — fetched 2026-09-03; source of the compaction.auto/prune/reserved descriptions and defaults quoted in §1.
• https://opencode.ai/docs/models/ — fetched 2026-09-03; confirms no compaction/context-limit content on that page (only reasoningEffort/thinking keys, unrelated to compaction).
• github.com/anomalyco/opencode @ b578b7261fc9ec4917fe272df5cc4bd8a056cd5d (cloned 2026-09-03, package.json version 1.18.27) — primary source for all source-code claims:
  • packages/opencode/src/session/overflow.ts — trigger formula (usable/isOverflow), §2
  • packages/opencode/src/session/compaction.ts — shipped compaction service, tail-turn selection, pruning, §2/§4
  • packages/opencode/src/session/session.ts (~lines 340–375) — usage → tokens mapping, reasoning-token split, §3
  • packages/core/src/v1/config/config.ts (~lines 95–170) — config schema, field descriptions, §1
  • packages/core/src/config/compaction.ts, packages/core/src/session/compaction.ts, packages/core/src/session/runner/llm.ts — the experimental/newer compaction engine, gated behind OPENCODE_EXPERIMENTAL_NATIVE_LLM, §2
  • packages/opencode/src/effect/runtime-flags.ts, packages/opencode/src/session/llm.ts (~line 226) — confirms which engine ships by default, §2
  • packages/opencode/src/provider/transform.ts (~line 1468, maxOutputTokens) — output-buffer sizing used in the reserved-token default, §2
• https://github.com/anomalyco/opencode/issues/11314 — "Configurable Context Compaction Threshold" (closed, not planned) — source of the "hardcoded 75%" community claim, §2/§4
• https://github.com/anomalyco/opencode/issues/11930 — "Configurable compaction threshold and model (global + per-model)" (closed, not planned) — source of the conflicting "100% threshold" claim, §2/§4
• https://github.com/anomalyco/opencode/issues/8089 — "Auto-compaction enabled by default, but context_length_exceeded errors still occur in agent workflows" (closed, not planned) — cited in §3 as a related-but-unconfirmed failure report
• https://github.com/anomalyco/opencode/issues/8140, #16375 — titles only, confirmed via WebSearch, not individually fetched; §4
• This repo: docs/research/opencode-cli-setup.md — structural template, and source of the limit.context/limit.output per-model config shape referenced throughout
• This repo: litellm-config.yaml — qwen3.8-27b-local model block, max_tokens comment on reasoning_content, §3

Confidence summary

Claim	Confidence
compaction is a single global top-level config key (auto/prune/reserved/tail_turns/preserve_recent_tokens)	High
No percentage-threshold config key exists	High
Trigger formula is usedTokens >= context − reservedBuffer, not a flat percentage	High
"Hardcoded 75%" (issue #11314) and "100% threshold" (issue #11930) are unverified community claims, not confirmed mechanism	High (that they're unverified/conflicting); the actual mechanism per source is definitive
overflow.ts/V1 path ships by default; newer core engine is gated behind OPENCODE_EXPERIMENTAL_NATIVE_LLM	High (default path); Medium (exact newer-engine behavior if enabled)
Reasoning tokens counted via usage.total_tokens in the normal (non-fallback) path	High
Reasoning tokens excluded from the fallback sum if total_tokens is ever absent	High (source); Medium (whether this repo's llama.cpp/litellm stack ever hits that fallback in practice)
No per-model/per-agent compaction threshold override exists; confirmed by rejected feature requests	High
Compaction mechanism is provider-agnostic — applies identically to @ai-sdk/openai-compatible custom providers	High
limit.context unset/0 on a custom model silently disables compaction for it	High