Compare commits

...
4 Commits
Author SHA1 Message Date
haylan aa89e07262 docs: record D9 review of 2.5 suggestions and done-log entries
check / check (push) Successful in 30s
2026-10-14 14:30:52 +02:00
haylan cc9a0ddf6e test: 2.5 orchestration run via local qwen3.8-27b (PASS)
check / check (push) Successful in 30s
2026-10-14 14:25:36 +02:00
haylan 14d6b2ce7a chore: prettier format
check / check (push) Successful in 40s
2026-10-14 12:49:05 +02:00
haylan 6d50e7903a docs: record 0.1.0 release, registry install verification and nightly proof
check / check (push) Failing after 31s
2026-10-14 12:48:52 +02:00
4 changed files with 137 additions and 23 deletions

No files matched your search

+12
View File
@@ -68,3 +68,15 @@ zero capability. Per-phase agents (spec-writer, ticket-planner, debugger, ...) r
Matt's phases share one conversation context by design (grilling facts feed to-spec; retro mines the
live session); isolating them would make the system worse. `triage` and `teach` agents rejected for
the same reason (continuous human dialogue belongs in the primary conversation).
## D9 — 2026-10-14 — Two 2.5 agent-improvement suggestions reviewed and rejected
The 2.5 orchestration run (PASS, commit `cc9a0dd`) suggested: (1) adding a line to `pocock.md`
about skill content looking truncated when a global `~/.agents/skills` install shadows the
plugin-injected id; (2) adding a reminder that a non-ff ticket merge is normal (resume the worker).
Both rejected: (1) is an artifact of one machine's pre-existing global install — on a clean machine
the package content is authoritative, and adding environment-quirk prose costs context load in
every session for zero general benefit (the model recovered unaided anyway); (2) is already the
documented path in `pocock.md` step 5, which the run followed correctly — duplicating it would
violate the no-duplicate-instructions validation criterion. Do not re-propose without evidence of
a run failing because of their absence.
+26 -16
View File
@@ -1,15 +1,20 @@
# Project status
Updated: 2026-10-14 (phase 1 implementation, ~90% done)
Updated: 2026-10-14 (2.5 orchestration run PASS — quality gate + end-to-end run both green)
## Current phase
Phase 1 (plugin package) — implementation complete and runtime-verified locally. Remaining:
first registry release (1.10) + Gitea secret provisioning, then Phase 2.
Phase 1 done AND 2.5 done: plugin installs, loads in OpenCode v2.0.29, skills inject, package
builds, `@haylan/opencode-pocock@0.1.0` published to the Gitea registry (release run 5684),
install-from-registry verified, nightly workflow proven (dispatch run 5685: github.com reachable,
no-change early exit), and a full manual orchestration run of the `pocock` orchestrator against a
3-ticket fixture (omniroute/qwen3.8-27b) is **PASS** — all 3 tickets built by parallel workers,
reviewed, fixed, merged, clean exit, no product bugs. Remaining: 2.7 (observe a week of
nightlies).
## Done
- Phase 0 analysis complete: architecture, 38-skill classification matrix, decisions D1–D8,
- Phase 0 analysis complete: architecture, 38-skill classification matrix, decisions D1–D9,
verified V2 API surface (docs + `@opencode/plugin@2.0.29` typings + live runtime).
- Package scaffolded: `@haylan/opencode-pocock` (name confirmed by owner), TS ESM, prettier,
`node --test` on .ts, MIT + NOTICE attributions (Matt Pocock, Dex Horthy, mcdays94, fastcrw).
@@ -38,21 +43,26 @@ first registry release (1.10) + Gitea secret provisioning, then Phase 2.
## Known gaps / risks
- 1.10 not done: package never published to the Gitea registry yet; npm-name plugin entry
(resolving through `exports`) unverified — local directory entry verified instead.
- `PACKAGE_TOKEN` Gitea secret must be provisioned by the repo admin before nightly can publish
or commit back.
- Nightly runner network access to github.com unverified (fallback: Gitea mirror of upstream).
- 2.5 DONE (2026-10-14): full orchestration run PASS on omniroute/qwen3.8-27b — see
docs/runtime-checks.md § "2.5 orchestration run". Noted env-only quirks: skill id collision
resolves `code-review` to the user's global `~/.agents/skills` copy (documented, not a plugin
bug); user's codebase-memory "graph-first" rule rejected one orchestrator `glob` (recovered via
`find`). Local 27b took ~63 min; reviewer subagents stream output only on completion (looked
stalled, was not). No product bugs found.
- 2.7: nightlies need a week of green runs before "stable" is declared; first scheduled run
2026-10-15 03:30 UTC.
- Background subagents in V2 (env-gated in V1) not verified; orchestrator text works either way.
- Windows-only smoke quirk: fixture temp dir deletion can lag (EPERM warning, harmless); CI is
Linux.
- Background subagents in V2 (env-gated in V1) not verified; orchestrator text works either way.
- `request.body` overlays accepted but not sent by the V2 runner — agents avoid them.
- Skill id collisions with pre-existing file-based installs resolve to the file copy (silent).
## Next steps (exact)
1. Run `npm run format`, `npm run check`, `npm test`, `npm run build`; commit workflows/smoke/docs.
2. Push to origin; ask owner to add `PACKAGE_TOKEN` secret (write:package + write:repository).
3. 1.10: `node scripts/release.ts` with SERVER_URL/REPOSITORY/SHA/TOKEN env (or dispatch a manual
release workflow) → verify `plugins: ["@haylan/opencode-pocock"]` installs from the registry in
a clean fixture; record in docs/runtime-checks.md.
4. Phase 2 per TODO.md (2.2 guardrails already implemented in 1.4 — verify mapping note; 2.4–2.7).
1. ~~2.5: manual orchestration run~~ DONE 2026-10-14 (PASS, see docs/runtime-checks.md).
2. Watch nightly runs for a week (2.7); on upstream release (17 pending changesets), verify the
full sync→generate→publish→commit-back cycle end-to-end.
3. Then declare stable and announce; keep classification.json current when upstream promotes
in-progress skills.
4. (Optional, owner) Decide whether to fold the 2.5 agent-improvement suggestions (truncated-skill
note, non-ff resume reminder) into agents/*.md — they are in the runtime-checks.md report.
+21 -7
View File
@@ -32,8 +32,11 @@ State: `.opencode/state/status.md`. Design: `docs/architecture.md`.
- [x] 1.8 .gitea/workflows/: check.yml, nightly.yml, smoke.yml (dispatch); scripts/smoke.ts —
PASS on v2.0.29 (plugin active, 31 skills, 16 hidden, agents + permissions decoded)
- [x] 1.9 local runtime verification → docs/runtime-checks.md (findings folded into README/AGENTS)
- [ ] 1.10 manual release 0.1.0 to Gitea registry; verify install-from-registry by package name
- [ ] ask repo admin to add Gitea secret PACKAGE_TOKEN (write:package + write:repository)
- [x] 1.10 manual release 0.1.0 to Gitea registry (dispatch run 5684, tag v0.1.0, package linked);
install-from-registry verified in clean fixture (plugin active as package source, npx init
works, anonymous registry reads) → docs/runtime-checks.md
- [x] ask repo admin to add Gitea secret PACKAGE_TOKEN — done 2026-10-14; nightly dispatch run
5685 proved runner github.com access + no-change early exit
## Phase 2 — conversion
@@ -45,8 +48,10 @@ State: `.opencode/state/status.md`. Design: `docs/architecture.md`.
permissions (edit/subagent/webfetch deny, git-read shell allow-list, skill allow code-review)
- [x] 2.4 orchestrator V2 flow wiring — done in 1.6: agent text covers background AND batch
dispatch; slash UX + /tmp default documented in README from runtime findings
- [ ] 2.5 per-skill regression checks (drift suite exists; add manual token-spending orchestration
- [x] 2.5 per-skill regression checks (drift suite exists; add manual token-spending orchestration
run: fixture spec + 3 tickets like pocock-agents smoke) + record in docs/runtime-checks.md
(DONE 2026-10-14: full PASS on omniroute/qwen3.8-27b, ~63 min, all 3 tickets + review + fixes,
no product bugs — see docs/runtime-checks.md § "2.5 orchestration run")
- [x] 2.6 README usage guide + finalize skill-mapping (matrix generated from classification.json)
- [ ] 2.7 one week of green nightlies -> declare stable (BLOCKED on PACKAGE_TOKEN secret)
@@ -59,11 +64,20 @@ State: `.opencode/state/status.md`. Design: `docs/architecture.md`.
- ~~Slash exposure for injected skills~~ RESOLVED: none (runtime check 2026-10-14); README
documents file-based alternatives.
- Package name/scope: CONFIRMED `@haylan/opencode-pocock` (owner, 2026-10-14).
- Does nightly runner have network access to github.com for upstream fetch? (1.8 — fallback: mirror
upstream to Gitea and sync from the mirror.) Verify on first nightly run.
- Install-from-registry by package name (npm-name plugin entry resolving through `exports`) —
verify at 1.10; local dir entry verified.
- ~~Does nightly runner have network access to github.com for upstream fetch?~~ VERIFIED
2026-10-14: nightly dispatch run 5685 success (clone + no-change early exit).
- ~~Install-from-registry by package name~~ VERIFIED 2026-10-14 (see docs/runtime-checks.md).
## Done log
- 2026-10-14: phase-0 analysis docs written (AGENTS.md, state/, docs/architecture.md, TODO.md).
- 2026-10-14: Phase 1 complete — 0.1.0 published to Gitea registry, install-from-registry verified,
nightly dispatch proven (runs 5684/5685).
- 2026-10-14: 2.5 orchestration run PASS on local omniroute/qwen3.8-27b (~63 min, 3/3 tickets,
16/16 tests, worktree loop + resume-worker merge path + review exercised, zero permission
escapes, zero product bugs) — commit cc9a0dd; suggestions reviewed → D9.
- 2026-10-14: 2.5 manual orchestration run — `pocock` orchestrator drove a 3-ticket `greeting`
fixture (omniroute/qwen3.8-27b, ~63 min) end-to-end: parallel workers on isolated worktrees,
non-ff resume path exercised, two-axis code review, review-fix worker, all 16 tests green on
`main`, worktrees cleaned, tickets marked done, `EXIT_CODE=0`. **PASS, no product bugs.**
Recorded in docs/runtime-checks.md § "2.5 orchestration run".
+78
View File
@@ -6,6 +6,17 @@ password printed on stdout) in a temp fixture and asserts against the HTTP API.
## Verified
- **Registry install by package name (1.10, 2026-10-14)**: `@haylan/opencode-pocock@0.1.0`
published to the Gitea npm registry by the dispatched `release` workflow (run 5684, success;
tag `v0.1.0` created, package linked to repo). Anonymous registry reads work (HTTP 200 without
auth). A clean fixture with `"plugins": ["@haylan/opencode-pocock"]` + scope registry in
`~/.npmrc` loads the plugin: `/api/plugin` shows
`source {type:"package", target:"@haylan/opencode-pocock", version:"0.1.0"}`,
`state.status "active"`; skills inject from the bun cache path
(`~/.cache/opencode/npm/@haylan/opencode-pocock@latest/.../skills/...`); `npx -y
@haylan/opencode-pocock init` installs the three agents (modes decode correctly).
Id collisions with a pre-existing global `~/.agents/skills` Matt Pocock install resolve to the
file-based copy (expected; a clean machine gets all 31 from the package).
- **Plugin loads and activates** from a local directory entry (`plugins: ["<abs path>/dist"]`);
`GET /api/plugin` shows `haylan.pocock` with `state.status: "active"`.
- A **file** entry (`file://.../dist/index.js`) is IGNORED — local plugin entries must be
@@ -44,3 +55,70 @@ password printed on stdout) in a temp fixture and asserts against the HTTP API.
harmless, retried on next run.
- The user's global `~/.agents/skills` already contains most Matt Pocock skills — smoke assertions
are written to tolerate id collisions (presence + flags, not source path).
## 2.5 orchestration run (2026-10-14, omniroute/qwen3.8-27b)
**Result: PASS** — full `implement-spec` flow, all 3 tickets, review, fixes, clean exit.
Fixture: fresh clone of the pocock-agents `greeting` reference (AGENTS.md `## Agent skills` block →
`docs/agents/issue-tracker.md` local tracker, GLOSSARY.md, `.scratch/greeting/spec.md` + 3 tickets —
01+02 independent, 03 blocked by 01+02, `src/greeting.js` + node:test, no remote, no deps). Agents
copied to `.opencode/agents/`; `opencode.json` =
`{"$schema":…,"plugins":["G:/_DEV/repos/mattpocock-skills-opencode/dist"],"model":"omniroute/qwen3.8-27b"}`
(`model` is a valid top-level v2 config key — confirmed against `opencode.ai/config.json`).
Command (cwd = fixture, background, `> run.log 2>&1`):
```
opencode run --agent pocock --auto "Implement the spec in .scratch/greeting/spec.md using
implement-spec: build all three tickets with parallel workers, review, and finish. The tracker is
local markdown (see docs/agents/issue-tracker.md). Do not ask me anything else; proceed through the
whole flow autonomously."
```
Duration: ~63 min (11:00:26Z → 12:03:47Z) on the local 27b model. `EXIT_CODE=0`.
| # | Check | Result | Evidence |
| --- | -------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | Integration + ticket branches exist and are merged | PASS | `spec/greeting` is an ancestor of `main`; `git merge-base --is-ancestor` confirms 01, 02, 03 all in `spec/greeting`. 01 + 03 ff-merged; 02 diverged after 01 landed → resolved via merge commit `0308c30 "Merge branch 'spec/greeting' into ticket/greeting/02"`, exactly the agent's documented resume-worker path. |
| 2 | Worktree dir used + cleaned | PASS | All worktrees under `/tmp/pocock-workers/pocock-25-fixture/greeting/` (`$WT` default). `git worktree list` shows only the main checkout after the run; orchestrator logged `git worktree remove --force … tickets/{01,02,03,review-fixes}` → `CLEANED`. Leftover empty `notes/` + `tickets/` dirs (no worktree admin entries) — harmless. |
| 3 | Tests pass on integration branch | PASS | Fresh worktree of `spec/greeting`: `npm test` = 16/16 pass, 0 fail (also 16/16 at every in-run merge point). |
| 4 | Flow evidence in run.log | PASS | `→ Skill "ask-matt"`, `→ Skill "implement-spec"`, `→ Skill "code-review"`; 4 `pocock-worker` dispatches (ticket 01/02 parallel, "Worker 02: merge integration tip" resume, ticket 03) + 4 `pocock-reviewer` dispatches (Spec axis, Standards axis, + compact re-runs); TDD red→green commit pairs per ticket (`test(…): red…` → `feat(…)`); no `✗` permission denials — zero worker escape attempts (no push/stash/out-of-allow-list skill). Only failure in the whole log: one `✗ Glob` hit by the user's global codebase-memory "graph-first" rule (environment quirk, below). |
| 5 | Model actually served | PASS | Every turn banner: `> pocock · qwen3.8-27b` — top-level `"model"` in opencode.json took effect for orchestrator and all nested subagents. |
| 6 | Ticket statuses updated | PASS | All three `issues/NN-*.md` now read `Status: done` with a `## Comments` note naming the branch — per the local-tracker convention in `docs/agents/issue-tracker.md`. |
Flow detail worth keeping: 02's divergence was handled by resuming worker 02 to merge the new
integration tip (agent text: "It knows its own change best, so it's the merger") — the non-ff path
worked, not just the happy ff path. Review produced 2 actionable Standards findings + 1 real Spec gap
(farewell error-message criterion untested); orchestrator dispatched a `review-fixes` worker for them
and merged back to `main`. `spec/greeting` then ff-merged to `main` (no remote → delivered on
default branch, per agent text).
Product bugs: **none**. Everything above is agent/plugin behaving per its documented design.
Findings / model-weakness notes (not product bugs):
- **Skill-id collision resolved to the user's global copy.** When the orchestrator loaded
`code-review`, the content came from `~/.agents/skills/code-review/SKILL.md` (pre-existing global
Matt Pocock install), not the plugin-injected copy — same collision documented in the 1.10 note.
The model then said the skill text was "truncated" and hunted for the SKILL.md on disk (found the
global one). Our plugin ships the full body (`code-review` body = 6177 chars, no slicing in
`src/index.ts`), so this is the collision + local-model behaviour, not truncation by the plugin.
- **`graph-first` rule collision.** The user's global codebase-memory-mcp injects a rule that forces a
graph query before any `glob`. One orchestrator `glob` was rejected with that error; the model
recovered by using `find`. Not our product; note for anyone running this with codebase-memory
installed.
- **Local 27b is slower than capable.** The whole run took ~63 min; reviewer subagents were CPU-busy
for ~14 min with no log growth (subagent output streams on completion, not incrementally — expected,
not a stall). No stalls requiring intervention; no permission denials; the only judgment shortcuts
were self-declared in the final summary (skipped a second post-fix review pass; left
spec-prescribed module duplication alone).
Suggested agent improvements (NOT applied — for the owner):
- `pocock.md`: consider noting that when a skill's content looks truncated, the model may re-read the
SKILL.md from disk; if the user also has a global Matt Pocock install, the id resolves to the file
copy — worth a one-line "skills may resolve to a global copy; content is authoritative either way".
- `pocock-worker.md`/`pocock.md`: the non-ff resume path works but is model-dependent; a short
reminder that "a ticket that can't ff-merge is normal when a parallel ticket merged first — resume
its worker" could make the 27b class of model take the documented path more reliably.