test: 2.5 orchestration run via local qwen3.8-27b (PASS)

.opencode/state/status.md

@@ -1,14 +1,16 @@
 # Project status
 
-Updated: 2026-10-14 (Phase 1 COMPLETE — quality gate passed)
+Updated: 2026-10-14 (2.5 orchestration run PASS — quality gate + end-to-end run both green)
 
 ## Current phase
 
-Phase 1 done: plugin installs, loads in OpenCode v2.0.29, skills inject, package builds,
+Phase 1 done AND 2.5 done: plugin installs, loads in OpenCode v2.0.29, skills inject, package
-`@haylan/opencode-pocock@0.1.0` published to the Gitea registry (release run 5684), install-from-
+builds, `@haylan/opencode-pocock@0.1.0` published to the Gitea registry (release run 5684),
-registry verified, nightly workflow proven (dispatch run 5685: github.com reachable, no-change
+install-from-registry verified, nightly workflow proven (dispatch run 5685: github.com reachable,
-early exit). Remaining: Phase 2.5 (manual token-spending orchestration run) and 2.7 (observe a
+no-change early exit), and a full manual orchestration run of the `pocock` orchestrator against a
-week of nightlies).
+3-ticket fixture (omniroute/qwen3.8-27b) is **PASS** — all 3 tickets built by parallel workers,
+reviewed, fixed, merged, clean exit, no product bugs. Remaining: 2.7 (observe a week of
+nightlies).
 
 ## Done
 
@@ -41,8 +43,12 @@ week of nightlies).
 
 ## Known gaps / risks
 
-- 2.5 not done: no token-spending end-to-end orchestration run yet (pocock building a fixture
+- 2.5 DONE (2026-10-14): full orchestration run PASS on omniroute/qwen3.8-27b — see
-  spec with parallel workers). Drift/unit/smoke suites cover the deterministic surface.
+  docs/runtime-checks.md § "2.5 orchestration run". Noted env-only quirks: skill id collision
+  resolves `code-review` to the user's global `~/.agents/skills` copy (documented, not a plugin
+  bug); user's codebase-memory "graph-first" rule rejected one orchestrator `glob` (recovered via
+  `find`). Local 27b took ~63 min; reviewer subagents stream output only on completion (looked
+  stalled, was not). No product bugs found.
 - 2.7: nightlies need a week of green runs before "stable" is declared; first scheduled run
   2026-10-15 03:30 UTC.
 - Background subagents in V2 (env-gated in V1) not verified; orchestrator text works either way.
@@ -53,10 +59,10 @@ week of nightlies).
 
 ## Next steps (exact)
 
-1. 2.5: manual orchestration run — fixture project with spec + 3 tickets (2 independent, 1
+1. ~~2.5: manual orchestration run~~ DONE 2026-10-14 (PASS, see docs/runtime-checks.md).
-   blocked), run `opencode --agent pocock`, verify worktree loop/merges/review/permissions;
-   record in docs/runtime-checks.md.
 2. Watch nightly runs for a week (2.7); on upstream release (17 pending changesets), verify the
    full sync→generate→publish→commit-back cycle end-to-end.
 3. Then declare stable and announce; keep classification.json current when upstream promotes
    in-progress skills.
+4. (Optional, owner) Decide whether to fold the 2.5 agent-improvement suggestions (truncated-skill
+   note, non-ff resume reminder) into agents/*.md — they are in the runtime-checks.md report.

TODO.md

@@ -48,8 +48,10 @@ State: `.opencode/state/status.md`. Design: `docs/architecture.md`.
       permissions (edit/subagent/webfetch deny, git-read shell allow-list, skill allow code-review)
 - [x] 2.4 orchestrator V2 flow wiring — done in 1.6: agent text covers background AND batch
       dispatch; slash UX + /tmp default documented in README from runtime findings
-- [ ] 2.5 per-skill regression checks (drift suite exists; add manual token-spending orchestration
+- [x] 2.5 per-skill regression checks (drift suite exists; add manual token-spending orchestration
       run: fixture spec + 3 tickets like pocock-agents smoke) + record in docs/runtime-checks.md
+      (DONE 2026-10-14: full PASS on omniroute/qwen3.8-27b, ~63 min, all 3 tickets + review + fixes,
+      no product bugs — see docs/runtime-checks.md § "2.5 orchestration run")
 - [x] 2.6 README usage guide + finalize skill-mapping (matrix generated from classification.json)
 - [ ] 2.7 one week of green nightlies -> declare stable (BLOCKED on PACKAGE_TOKEN secret)
 
@@ -69,3 +71,8 @@ State: `.opencode/state/status.md`. Design: `docs/architecture.md`.
 ## Done log
 
 - 2026-10-14: phase-0 analysis docs written (AGENTS.md, state/, docs/architecture.md, TODO.md).
+- 2026-10-14: 2.5 manual orchestration run — `pocock` orchestrator drove a 3-ticket `greeting`
+  fixture (omniroute/qwen3.8-27b, ~63 min) end-to-end: parallel workers on isolated worktrees,
+  non-ff resume path exercised, two-axis code review, review-fix worker, all 16 tests green on
+  `main`, worktrees cleaned, tickets marked done, `EXIT_CODE=0`. **PASS, no product bugs.**
+  Recorded in docs/runtime-checks.md § "2.5 orchestration run".

docs/runtime-checks.md

@@ -55,3 +55,70 @@ password printed on stdout) in a temp fixture and asserts against the HTTP API.
   harmless, retried on next run.
 - The user's global `~/.agents/skills` already contains most Matt Pocock skills — smoke assertions
   are written to tolerate id collisions (presence + flags, not source path).
+
+## 2.5 orchestration run (2026-10-14, omniroute/qwen3.8-27b)
+
+**Result: PASS** — full `implement-spec` flow, all 3 tickets, review, fixes, clean exit.
+
+Fixture: fresh clone of the pocock-agents `greeting` reference (AGENTS.md `## Agent skills` block →
+`docs/agents/issue-tracker.md` local tracker, GLOSSARY.md, `.scratch/greeting/spec.md` + 3 tickets —
+01+02 independent, 03 blocked by 01+02, `src/greeting.js` + node:test, no remote, no deps). Agents
+copied to `.opencode/agents/`; `opencode.json` =
+`{"$schema":…,"plugins":["G:/_DEV/repos/mattpocock-skills-opencode/dist"],"model":"omniroute/qwen3.8-27b"}`
+(`model` is a valid top-level v2 config key — confirmed against `opencode.ai/config.json`).
+
+Command (cwd = fixture, background, `> run.log 2>&1`):
+
+```
+opencode run --agent pocock --auto "Implement the spec in .scratch/greeting/spec.md using
+implement-spec: build all three tickets with parallel workers, review, and finish. The tracker is
+local markdown (see docs/agents/issue-tracker.md). Do not ask me anything else; proceed through the
+whole flow autonomously."
+```
+
+Duration: ~63 min (11:00:26Z → 12:03:47Z) on the local 27b model. `EXIT_CODE=0`.
+
+| #   | Check                                              | Result | Evidence                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+| --- | -------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| 1   | Integration + ticket branches exist and are merged | PASS   | `spec/greeting` is an ancestor of `main`; `git merge-base --is-ancestor` confirms 01, 02, 03 all in `spec/greeting`. 01 + 03 ff-merged; 02 diverged after 01 landed → resolved via merge commit `0308c30 "Merge branch 'spec/greeting' into ticket/greeting/02"`, exactly the agent's documented resume-worker path.                                                                                                                                                                                                                                                             |
+| 2   | Worktree dir used + cleaned                        | PASS   | All worktrees under `/tmp/pocock-workers/pocock-25-fixture/greeting/` (`$WT` default). `git worktree list` shows only the main checkout after the run; orchestrator logged `git worktree remove --force … tickets/{01,02,03,review-fixes}` → `CLEANED`. Leftover empty `notes/` + `tickets/` dirs (no worktree admin entries) — harmless.                                                                                                                                                                                                                                        |
+| 3   | Tests pass on integration branch                   | PASS   | Fresh worktree of `spec/greeting`: `npm test` = 16/16 pass, 0 fail (also 16/16 at every in-run merge point).                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+| 4   | Flow evidence in run.log                           | PASS   | `→ Skill "ask-matt"`, `→ Skill "implement-spec"`, `→ Skill "code-review"`; 4 `pocock-worker` dispatches (ticket 01/02 parallel, "Worker 02: merge integration tip" resume, ticket 03) + 4 `pocock-reviewer` dispatches (Spec axis, Standards axis, + compact re-runs); TDD red→green commit pairs per ticket (`test(…): red…` → `feat(…)`); no `✗` permission denials — zero worker escape attempts (no push/stash/out-of-allow-list skill). Only failure in the whole log: one `✗ Glob` hit by the user's global codebase-memory "graph-first" rule (environment quirk, below). |
+| 5   | Model actually served                              | PASS   | Every turn banner: `> pocock · qwen3.8-27b` — top-level `"model"` in opencode.json took effect for orchestrator and all nested subagents.                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+| 6   | Ticket statuses updated                            | PASS   | All three `issues/NN-*.md` now read `Status: done` with a `## Comments` note naming the branch — per the local-tracker convention in `docs/agents/issue-tracker.md`.                                                                                                                                                                                                                                                                                                                                                                                                             |
+
+Flow detail worth keeping: 02's divergence was handled by resuming worker 02 to merge the new
+integration tip (agent text: "It knows its own change best, so it's the merger") — the non-ff path
+worked, not just the happy ff path. Review produced 2 actionable Standards findings + 1 real Spec gap
+(farewell error-message criterion untested); orchestrator dispatched a `review-fixes` worker for them
+and merged back to `main`. `spec/greeting` then ff-merged to `main` (no remote → delivered on
+default branch, per agent text).
+
+Product bugs: **none**. Everything above is agent/plugin behaving per its documented design.
+
+Findings / model-weakness notes (not product bugs):
+
+- **Skill-id collision resolved to the user's global copy.** When the orchestrator loaded
+  `code-review`, the content came from `~/.agents/skills/code-review/SKILL.md` (pre-existing global
+  Matt Pocock install), not the plugin-injected copy — same collision documented in the 1.10 note.
+  The model then said the skill text was "truncated" and hunted for the SKILL.md on disk (found the
+  global one). Our plugin ships the full body (`code-review` body = 6177 chars, no slicing in
+  `src/index.ts`), so this is the collision + local-model behaviour, not truncation by the plugin.
+- **`graph-first` rule collision.** The user's global codebase-memory-mcp injects a rule that forces a
+  graph query before any `glob`. One orchestrator `glob` was rejected with that error; the model
+  recovered by using `find`. Not our product; note for anyone running this with codebase-memory
+  installed.
+- **Local 27b is slower than capable.** The whole run took ~63 min; reviewer subagents were CPU-busy
+  for ~14 min with no log growth (subagent output streams on completion, not incrementally — expected,
+  not a stall). No stalls requiring intervention; no permission denials; the only judgment shortcuts
+  were self-declared in the final summary (skipped a second post-fix review pass; left
+  spec-prescribed module duplication alone).
+
+Suggested agent improvements (NOT applied — for the owner):
+
+- `pocock.md`: consider noting that when a skill's content looks truncated, the model may re-read the
+  SKILL.md from disk; if the user also has a global Matt Pocock install, the id resolves to the file
+  copy — worth a one-line "skills may resolve to a global copy; content is authoritative either way".
+- `pocock-worker.md`/`pocock.md`: the non-ff resume path works but is model-dependent; a short
+  reminder that "a ticket that can't ff-merge is normal when a parallel ticket merged first — resume
+  its worker" could make the 27b class of model take the documented path more reliably.