chore: add project docs, glossary, research and opencode/fastcrw skills

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 17:30:03 +02:00
commit b316daed43
@@ -0,0 +1,53 @@
 ---
 name: fastcrw-search-api
 description: Contract of the fastCRW `POST /v1/search` REST endpoint (auth, request, response shapes, errors, env vars). Use when writing or testing the plugin's search call or its response/error mapping.
 ---
 # fastCRW `/v1/search`
 Researched from docs.fastcrw.com and github.com/us/crw source; not tested against a live instance. Full detail: `docs/research/fastcrw-search-api.md`.
 ## Call
 `POST {baseURL}/v1/search`, `Content-Type: application/json`, body `{ "query": string, "limit": number }`.
 - `query` required, max 2000 chars. `limit` default 5, clamped server-side to 1..20.
 - Optional extras worth exposing: `lang`, `tbs` (`qdr:h|d|w|m|y`). Do not send `sources` (switches the response to a grouped shape).
 ## Auth
 `Authorization: Bearer <key>`. Send it only when a key is configured; self-hosted needs none unless the operator set `CRW_AUTH__API_KEYS`. 401 messages: "Missing Authorization header", "Invalid API key".
 ## Env vars (match upstream names)
 `CRW_API_URL` (base URL; hosted default `https://api.fastcrw.com`, self-hosted `http://localhost:3000`), `CRW_API_KEY` (token).
 ## Response: parse both shapes
 ```json
 { "success": true, "data": [ ... ] }
 { "success": true, "data": { "results": [ ... ] } }
 ```
 Result: `url`, `title`, `description`, `snippet` (always-populated alias of `description`), `position`, optional `score`, `publishedDate`, `category`. Map to opencode as `{ url, title, content: snippet || description || "", time: {} }`.
 ## Errors
 Body `{ "success": false, "error": string, "error_code": string }`.
 | HTTP | error_code |
 |---|---|
 | 400 | `invalid_request` |
 | 401 | auth middleware message |
 | 422 | `target_unreachable` |
 | 429 | `rate_limited` |
 | 500 | `internal_error` |
 | 502 | `http_error` |
 | 503 | `search_disabled` / `search_degraded` |
 | 504 | `timeout` |
 Self-hosted search needs a SearXNG-style backend (`CRW_SEARCH__SEARCH_BACKEND_URL`); without one expect 503 `search_disabled`.
 ## Limits
 Server defaults: 10 req/s, 60 s request timeout, 15 s search timeout. A plain `fetch` suffices; `crw-mcp` is unnecessary for search only.
@@ -0,0 +1,65 @@
 ---
 name: opencode-v2-plugin
 description: How to write, configure and register an opencode v2 plugin, including adding a websearch provider. Use when writing or editing plugin code, opencode.json `plugins`/`websearch` config, or plugin packaging in this repo.
 ---
 # opencode v2 plugin
 Verified against opencode.ai/v2 docs (build/plugins, plugins, config, websearch). Full evidence and open items: `docs/research/opencode-v2-plugin-api.md`. The v1 package `@opencode-ai/plugin` does NOT run in v2.
 ## Entry point
 ```ts
 import { Plugin } from "@opencode/plugin"
 export default Plugin.define({
  id: "home.fastcrw", // required, stable; used for storage scoping and diagnostics
  async setup(ctx) {
    // ctx.options (plugin options), ctx.storage, ctx.location, ctx.websearch, ...
    // may return a cleanup function
  },
 })
 ```
 Options are read from `ctx.options` inside `setup`, not passed as an argument. They are untyped key/value: validate them yourself.
 ## Websearch provider
 ```ts
 await ctx.websearch.transform((editor) => {
  editor.add({
    id: "fastcrw",
    name: "fastCRW (self-hosted)",
    execute: async ({ query }, { signal }) => [{ url, title, content, time: {} }],
  })
  editor.default.set("fastcrw") // make it active; set(false) disables
 })
 ```
 - `execute` returns `{ url, title, content, time }[]`. Pass `signal` to `fetch`.
 - `await ctx.websearch.reload()` re-evaluates providers.
 ## Config (`opencode.json`)
 ```jsonc
 {
  "plugins": [{ "package": "~/.config/opencode/plugins/fastcrw", "options": { "baseURL": "http://crw.home:3000", "limit": 8 } }],
  "websearch": { "provider": "fastcrw" }
 }
 ```
 - Key is plural `plugins` (v1: `plugin`). Entries are a string or `{ package, options }`.
 - `package` may be an npm name (`name`, `name@1.2.0`, `@scope/name`), `./local`, an absolute path or `file://`.
 - Auto-discovered: `.ts`/`.js` files and packages in `.opencode/plugins/` and `~/.config/opencode/plugins/`.
 - `websearch.provider` is a provider id or `"random"`; `"websearch": false` disables search. Built-in ids: exa, firecrawl, parallel, tavily, tinyfish.
 - CLI: `opencode plugin add|list|check|update|remove`.
 ## Packaging (npm)
 `"type": "module"`, `exports["."]` pointing at the entry, dependency on `@opencode/plugin`.
 ## Unverified: check at runtime, do not assume
 - Exact TypeScript types for the editor, `time` and `execute` args (read `@opencode/plugin` typings after install).
 - Inner shape of `time`, and whether extra result fields are allowed.
 - Precedence between config `websearch.provider` and `editor.default.set(id)`.
@@ -0,0 +1,13 @@
 ## Agent skills
 ### Issue tracker
 Issues are tracked in Gitea (git.arthurerlich.de) via the `tea` CLI. See `docs/agents/issue-tracker.md`.
 ### Triage labels
 Default vocabulary: `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`. See `docs/agents/triage-labels.md`.
 ### Domain docs
 Single-context: one `CONTEXT.md` + `docs/adr/` at the repo root. See `docs/agents/domain.md`.
@@ -0,0 +1,25 @@
 # fastCRW websearch plugin for opencode
 An opencode v2 plugin that makes a self-hosted fastCRW server opencode's websearch.
 ## Language
 **Plugin**:
 The opencode v2 module this repo builds and ships.
 _Avoid_: extension, integration
 **Search provider**:
 A named websearch backend that opencode can select. This plugin registers one, `fastcrw`, and makes it the only one.
 _Avoid_: engine, backend (a fastCRW server's own search backend is a different thing)
 **Endpoint**:
 The base URL of the fastCRW server the plugin talks to. Set by the `baseURL` option or `CRW_API_URL`; defaults to `http://localhost:3000`.
 _Avoid_: host, server URL
 **Token**:
 The optional Bearer key sent to the fastCRW server. Set by the `apiKey` option or `CRW_API_KEY`. Absent means no auth header.
 _Avoid_: secret, password
 **Plugin options**:
 The values opencode passes to the plugin from `opencode.json`: `baseURL`, `apiKey`, `limit`, `lang`, `tbs`.
 _Avoid_: settings, config (config means `opencode.json` as a whole)
@@ -0,0 +1,35 @@
 # Domain Docs
 How the engineering skills should consume this repo's domain documentation when exploring the codebase.
 ## Before exploring, read these
 - **`CONTEXT.md`** at the repo root
 - **`docs/adr/`** — read ADRs that touch the area you're about to work in.
 If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The `/domain-modeling` skill creates them lazily when terms or decisions actually get resolved.
 ## File structure
 Single-context repo:
 ```
 /
 ├── CONTEXT.md
 ├── docs/adr/
 │   ├── 0001-event-sourced-orders.md
 │   └── 0002-postgres-for-write-model.md
 └── src/
 ```
 ## Use the glossary's vocabulary
 When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids.
 If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/domain-modeling`).
 ## Flag ADR conflicts
 If your output contradicts an existing ADR, surface it explicitly rather than silently overriding:
 > _Contradicts ADR-0007 (event-sourced orders) — but worth reopening because…_
@@ -0,0 +1,26 @@
 # Issue tracker: Gitea
 Issues and specs for this repo live as Gitea issues on `git.arthurerlich.de`. Use the `tea` CLI for all operations (see the `gitea-tea` skill).
 ## Conventions
 - **Create an issue**: `tea issues create --title "..." --description "..."`. Use a heredoc for multi-line bodies.
 - **Read an issue**: `tea issues <number>` (shows comments with `--comments`).
 - **List issues**: `tea issues list --state open --fields index,title,body,labels` with `--labels` filters as needed.
 - **Comment on an issue**: `tea comment <number> "..."`
 - **Apply / remove labels**: `tea issues edit <number> --add-labels "..."` / `--remove-labels "..."`
 - **Close**: `tea issues close <number>` (add a comment first with `tea comment`).
 Infer the repo from `git remote -v` — `tea` does this automatically when run inside a clone with a configured login. Run `tea --help` / `tea issues --help` if a flag differs in your installed version.
 ## Pull requests as a triage surface
 **PRs as a request surface: no.** _(Set to `yes` if this repo treats external PRs as feature requests.)_
 ## When a skill says "publish to the issue tracker"
 Create a Gitea issue.
 ## When a skill says "fetch the relevant ticket"
 Run `tea issues <number>` with comments.
@@ -0,0 +1,15 @@
 # Triage Labels
 The skills speak in terms of five canonical triage roles. This file maps those roles to the actual label strings used in this repo's issue tracker.
 | Label in mattpocock/skills | Label in our tracker | Meaning                                  |
 | -------------------------- | -------------------- | ---------------------------------------- |
 | `needs-triage`             | `needs-triage`       | Maintainer needs to evaluate this issue  |
 | `needs-info`               | `needs-info`         | Waiting on reporter for more information |
 | `ready-for-agent`          | `ready-for-agent`    | Fully specified, ready for an AFK agent  |
 | `ready-for-human`          | `ready-for-human`    | Requires human implementation            |
 | `wontfix`                  | `wontfix`            | Will not be actioned                     |
 When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table.
 Edit the right-hand column to match whatever vocabulary you actually use.
@@ -0,0 +1,102 @@
 # fastCRW `POST /v1/search` contract
 Resolves issue #3. Researched 2026-10-03 against `main` of github.com/us/crw (`fastcrw/crw`), docs.fastcrw.com and npm `crw-mcp` 0.37.2.
 ## TL;DR for the plugin
 - Endpoint: `POST {CRW_API_URL}/v1/search`, JSON body `{"query": "...", "limit": N}`.
 - Auth: `Authorization: Bearer $CRW_API_KEY` for hosted. Self-hosted sends no header unless the operator set `CRW_AUTH__API_KEYS`; then it is the same Bearer scheme.
 - **Parse defensively: `data` is a flat array on hosted, but `{"data": {"results": [...]}}` on self-hosted.** The official CLI accepts both. When `sources` is set, results are grouped as `{web:[],news:[],images:[]}` instead of an array.
 - Every result has both `description` and `snippet` (snippet is an always-populated alias of description).
 - Errors: `{"success": false, "error": "...", "error_code": "..."}` with HTTP 4xx/5xx.
 ## Auth
 - Hosted (`https://api.fastcrw.com`): `Authorization: Bearer <key>` required. [docs.fastcrw.com REST API]
 - Self-hosted (`http://localhost:3000`): no header needed by default. Middleware bypasses auth when `api_keys` is empty; otherwise checks the `Bearer ` token with constant-time compare. 401 messages: `Missing Authorization header`, `Invalid API key`. [crates/crw-server/src/middleware.rs]
 - Server-side keys are set via `CRW_AUTH__API_KEYS` (comma-separated or JSON array). [crates/crw-core/src/config.rs `AuthConfig`]
 ## Env var names
 | Var | Used by | Meaning |
 |---|---|---|
 | `CRW_API_URL` | crw-mcp, crw CLI | Remote server base URL. Default `https://api.fastcrw.com`; `http://localhost:3000` for self-hosted. |
 | `CRW_API_KEY` | crw-mcp, crw CLI | Bearer token for the remote server. |
 | `CRW_LOCAL` | crw-mcp | Truthy forces embedded mode, ignoring `CRW_API_URL`. |
 | `CRW_AUTH__API_KEYS` | server | Accepted keys (server side). |
 | `CRW_SEARCH__*` | server | Search config (`enabled`, `search_backend_url`, `timeout_ms`, `default_limit`, `max_limit`). |
 | `CRW_SEARXNG_URL` | CLI | Legacy local search backend URL. |
 Sources: docs.fastcrw.com (REST API section), crates/crw-core/src/config.rs, crw-mcp README.
 Plugin should use `CRW_API_URL` / `CRW_API_KEY` to match upstream naming.
 ## Request body
 Source: `SearchRequest` in crates/crw-core/src/types.rs (camelCase serde); blog/crw-search-api-release.md.
 | Field | Type | Notes |
 |---|---|---|
 | `query` | string, required | Max 2000 chars (handler validation). |
 | `limit` | int, optional | Default 5 (`default_limit`); clamped server-side to `1..=max_limit` (max 20). |
 | `lang` | string | SearXNG language, e.g. `en`, `de`, `auto`; validated as a language tag. |
 | `tbs` | enum | `qdr:h`, `qdr:d`, `qdr:w`, `qdr:m`, `qdr:y`. |
 | `sources` | array | `web`, `news`, `images`. Presence switches response to grouped form. |
 | `categories` | array, max 5 | `github`, `research`, `pdf`, or passthrough string. |
 | `scrapeOptions` | object | Scrapes each web result (timeout bounds 1..60000 ms). Adds `markdown` etc. to results. |
 | `answer`, `answerTopN`, `summarizeResults`, `maxCharsPerSource`, `llmApiKey`, `llmProvider`, `llmModel` | | LLM synthesis; need `scrapeOptions`. Not needed by this plugin. |
 Minimal request is `{"query","limit"}`; `lang`/`tbs` are the only extras worth exposing.
 ## Response
 Envelope `ApiResponse` (camelCase, absent fields omitted): `success`, `data`, `error`, `error_code`, `code` (v2 compat only), `warning`.
 `data` is `SearchResponseData` = `{results, answer?, citations?, llmUsage?, warnings?}`. `results` is untagged: flat `SearchResult[]` or grouped `{web,news,images}`.
 Hosted example (flat, per docs):
 ```json
 {"success": true, "data": [
  {"url": "https://...", "title": "...", "description": "...", "snippet": "...", "position": 1, "score": 9.5}
 ]}
 ```
 Self-hosted (CLI parser handles `{"success": true, "data": {"results": [ ... ]}}`).
 `SearchResult`: `url`, `title`, `description`, `snippet` (alias of description, always populated), `position` (u32), optional `score`, `publishedDate`, `category`, and (with scrapeOptions) `markdown`, `html`, `rawHtml`, `links`.
 Conflict note: the docs example and CLI show flat `data[]` for hosted; the struct and CLI show `data.results` for the server. I did not call a live instance, so verify against the target server and keep the dual-shape parser.
 ## Errors
 JSON body `{"success": false, "error": "<message>", "error_code": "<code>"}` (crates/crw-server/src/error.rs).
 | HTTP | error_code | Cause |
 |---|---|---|
 | 400 | `invalid_request` | bad query/limit/lang |
 | 401 | (auth middleware message) | missing/invalid key |
 | 429 | `rate_limited` | rate limit |
 | 502 | `http_error` | search backend HTTP/JSON failure |
 | 504 | `timeout` | search timeout (default 15 s) |
 | 503 | `search_disabled` / `search_degraded` | search off or backend unhealthy |
 | 422 | `target_unreachable` | backend unreachable |
 | 500 | `internal_error` | other |
 Backend hostnames/credentials are never leaked in errors.
 ## Limits
 - `limit` max 20 (default 5); `query` max 2000 chars; `categories` max 5.
 - Server defaults: `rate_limit_rps` 10 (0 = unlimited), `request_timeout_secs` 60, search `timeout_ms` 15000.
 - Hosted credits: 1 per search, +1 per scraped result. Rate/size limits of hosted plan are not documented in sources found.
 - Self-hosted: search only works when a SearXNG-style backend is configured (`CRW_SEARCH__SEARCH_BACKEND_URL`); Docker Compose ships a search sidecar, the bare single binary does not. If unconfigured, expect 503 `search_disabled`. (An older blog post claimed self-hosted had no search; the current config.rs/self-hosting docs contradict it.)
 ## crw-mcp vs REST
 crw-mcp 0.37.2 exposes 9 tools (`crw_scrape`, `crw_crawl`, `crw_check_crawl_status`, `crw_map`, `crw_extract`, `crw_check_extract_status`, `crw_cancel_extract`, `crw_search`, `crw_parse_file`). In proxy mode (`CRW_API_URL` set) it just forwards to the REST API with the Bearer key; in embedded mode `crw_search` only appears if a backend is configured. The REST call makes the MCP layer, its Node/binary install and the other 8 tools unnecessary for a search-only plugin; a plain `fetch` to `/v1/search` suffices.
 ## Sources
 - https://docs.fastcrw.com/ (REST API), /self-hosting/, /mcp-clients/
 - https://github.com/us/crw: crates/crw-core/src/{types.rs,config.rs,error.rs}, crates/crw-server/src/{middleware.rs,error.rs,routes/search.rs}, crates/crw-cli/src/commands/search.rs, crates/crw-mcp README, blog/crw-search-api-release.md
 - https://registry.npmjs.org/crw-mcp
@@ -0,0 +1,75 @@
 # Research: verified opencode v2 plugin API (issue #2)
 Researched 2026-10-03. Sources (all first-party opencode docs):
 - [S1] https://opencode.ai/v2/docs/build/plugins (build a plugin)
 - [S2] https://opencode.ai/v2/docs/plugins (configure plugins)
 - [S3] https://opencode.ai/v2/docs/config
 - [S4] https://opencode.ai/v2/docs/websearch
 - [S5] https://opencode.ai/v2/docs/build/plugins/migrate-v1
 - [S6] https://opencode.ai/v2/docs/api (HTTP API reference)
 Caveat: pages were read through a fetch-and-summarise tool, so quotes are as returned by it, not byte-checked. The opencode GitHub source (anomalyco/opencode) was NOT inspected (code search needs login); anything needing exact TypeScript types is flagged "unverified" below.
 ## Verdict on the user's sketch
 | Sketch element | Verdict | Evidence |
 |---|---|---|
 | `Plugin.define` from `"@opencode/plugin"` | RIGHT | [S1] `import { Plugin } from "@opencode/plugin"` / `export default Plugin.define({ id: "example", async setup(ctx) {...} })`. V1 package was `@opencode-ai/plugin` [S5]. |
 | `ctx.websearch.transform` | RIGHT | [S1] `await ctx.websearch.transform((editor) => { ... })` |
 | `editor.add` | RIGHT | [S1] `editor.add({ id: "internal", name: "Internal search", execute: async ({ query }, { signal }) => {...} })` |
 | `plugins[].package` / `options` in opencode.json | RIGHT | [S2][S3] key is `"plugins"` (V1 was `"plugin"`, tuples replaced by objects [S5]); object form `{ "package": "...", "options": {...} }` |
 | `websearch.provider` | RIGHT (config key) | [S3][S4] `{ "websearch": { "provider": "tavily" } }`; also `"random"`. Built-in ids: exa, firecrawl, parallel, tavily, tinyfish [S4]. |
 Things the sketch likely missed / must add:
 - `id` is required on `Plugin.define` (stable, used for storage scoping/diagnostics) [S5].
 - A plugin-added provider is made active with `editor.default.set("<id>")` [S1]; `websearch.provider` in config selects by provider id, so `"provider": "<your-id>"` is the expected pairing (inference from [S1]+[S4]; docs do not state it explicitly -> verify at runtime).
 - Options are read as `ctx.options` (not passed as a setup arg) [S1].
 ## Sub-questions
 ### 1. Plugin entry / definition shape
 Each module has one default export with a unique id and `setup`:
 ```ts
 import { Plugin } from "@opencode/plugin"
 export default Plugin.define({
  id: "example",
  async setup(ctx) { /* may return a cleanup function */ },
 })
 ```
 [S1][S5]. Context: `ctx.app.version`, `ctx.location` (`directory`, optional `workspaceID`, project), `ctx.options`, `ctx.storage` (durable JSON, plugin-scoped), plus domains `ctx.session/permission/shell/tool/websearch/...` [S1][S5]. Return cleanup from `setup` for timers/sockets [S5]. V1 plugins do not run in V2; dual-support is possible via spreading `Plugin.define(...)` and adding a V1 `server()` [S1][S5].
 ### 2. Registration/loading (local path vs npm)
 `"plugins"` array in `opencode.json(c)`; entries are strings or objects [S2]:
 npm `"opencode-acme-plugin"`, `"opencode-acme-plugin@1.2.0"`, `"@acme/opencode-plugin"`; local `"./plugins/local"`, `"../shared/plugin.ts"`, `"/abs/plugin.ts"`, `"file:///..."`. Also auto-discovery of `.ts/.js` files and packages in `.opencode/plugins/` or `~/.config/opencode/plugins/` [S2]. CLI: `opencode plugin add|list|check|update|remove` [S2]. Disable via ids: `["*", "-opencode.provider.*", ...]` [S2]. Published package needs `"type":"module"`, `exports["."]` -> entry, dependency `@opencode/plugin` [S1].
 ### 3. Passing options
 ```jsonc
 { "plugins": [ { "package": "./plugins/company", "options": { "strict": true } } ] }
 ```
 read via `ctx.options.strict` in `setup` [S1][S3]. Options type is untyped key/value (validate yourself).
 ### 4. Can a plugin provide/replace the websearch provider?
 Yes. [S1]:
 ```ts
 await ctx.websearch.transform((editor) => {
  editor.add({ id: "internal", name: "Internal search",
    execute: async ({ query }, { signal }) => {
      const response = await fetch(`https://search.example.com?q=${query}`, { signal })
      return [{ url: "", title: "", content: "", time: {} }]
    } })
  editor.default.set("internal")
 })
 ```
 Disable: `editor.default.set(false)`; re-evaluate: `await ctx.websearch.reload()` [S1]. The API reference lists add/default/remove operations [S6] (remove not shown in S1 example; exact editor type unverified).
 ### 5. How `websearch.provider` is selected
 Config `{ "websearch": { "provider": "<id>" | "random" } }`; `"random"` picks an available provider; `"websearch": false` disables search entirely [S3][S4]. Provider on HTTP 429 enters cooldown and search retries another provider [S4]. Provider API keys via `/connect` or env (e.g. `TAVILY_API_KEY`) [S4]. For a plugin provider, `editor.default.set(id)` is the plugin-side selection [S1]; interplay/precedence with the config `provider` key is not documented -> verify.
 ### 6. Required search result shape
 From [S1] the `execute` return is an array of `{ url, title, content, time }`; the example uses `time: {}` (object, inner shape undocumented). [S6] lists the same four fields. Unverified: whether `time` fields are optional, exact `time` keys (e.g. published/updated), and whether extra fields are allowed. `execute` receives `({ query }, { signal })`; `signal` is an AbortSignal and must be passed to fetch [S1].
 ## Open items (need source or runtime check)
 - Exact TS types for editor, `time`, and execute args (read anomalyco/opencode source or `@opencode/plugin` typings after install).
 - Precedence of config `websearch.provider` vs plugin `editor.default.set`.
 - [S6] suggested `POST /api/websearch/query` with `{query, provider?}` -> `{results, provider}`; this came from the summariser and is lower confidence.