haylan/fastcrw-opencode-websearch
1
0 Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

chore: add TypeScript, Prettier and changelog scaffold

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
haylanandClaude Bot committed 2026-10-03 19:04:21 +02:00
1 parent 6da76a8a18
commit 5e3671fa8b
12 files changed
+543 -70
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/research/fastcrw-search-api.md
+41 -31
View File
@@ -18,14 +18,14 @@ Resolves issue #3. Researched 2026-10-03 against `main` of github.com/us/crw (`f
## Env var names
| Var | Used by | Meaning |
|---|---|---|
| `CRW_API_URL` | crw-mcp, crw CLI | Remote server base URL. Default `https://api.fastcrw.com`; `http://localhost:3000` for self-hosted. |
| `CRW_API_KEY` | crw-mcp, crw CLI | Bearer token for the remote server. |
| `CRW_LOCAL` | crw-mcp | Truthy forces embedded mode, ignoring `CRW_API_URL`. |
| `CRW_AUTH__API_KEYS` | server | Accepted keys (server side). |
| `CRW_SEARCH__*` | server | Search config (`enabled`, `search_backend_url`, `timeout_ms`, `default_limit`, `max_limit`). |
| `CRW_SEARXNG_URL` | CLI | Legacy local search backend URL. |
| Var | Used by | Meaning |
| -------------------- | ---------------- | --------------------------------------------------------------------------------------------------- |
| `CRW_API_URL` | crw-mcp, crw CLI | Remote server base URL. Default `https://api.fastcrw.com`; `http://localhost:3000` for self-hosted. |
| `CRW_API_KEY` | crw-mcp, crw CLI | Bearer token for the remote server. |
| `CRW_LOCAL` | crw-mcp | Truthy forces embedded mode, ignoring `CRW_API_URL`. |
| `CRW_AUTH__API_KEYS` | server | Accepted keys (server side). |
| `CRW_SEARCH__*` | server | Search config (`enabled`, `search_backend_url`, `timeout_ms`, `default_limit`, `max_limit`). |
| `CRW_SEARXNG_URL` | CLI | Legacy local search backend URL. |
Sources: docs.fastcrw.com (REST API section), crates/crw-core/src/config.rs, crw-mcp README.
Plugin should use `CRW_API_URL` / `CRW_API_KEY` to match upstream naming.
@@ -34,16 +34,16 @@ Plugin should use `CRW_API_URL` / `CRW_API_KEY` to match upstream naming.
Source: `SearchRequest` in crates/crw-core/src/types.rs (camelCase serde); blog/crw-search-api-release.md.
| Field | Type | Notes |
|---|---|---|
| `query` | string, required | Max 2000 chars (handler validation). |
| `limit` | int, optional | Default 5 (`default_limit`); clamped server-side to `1..=max_limit` (max 20). |
| `lang` | string | SearXNG language, e.g. `en`, `de`, `auto`; validated as a language tag. |
| `tbs` | enum | `qdr:h`, `qdr:d`, `qdr:w`, `qdr:m`, `qdr:y`. |
| `sources` | array | `web`, `news`, `images`. Presence switches response to grouped form. |
| `categories` | array, max 5 | `github`, `research`, `pdf`, or passthrough string. |
| `scrapeOptions` | object | Scrapes each web result (timeout bounds 1..60000 ms). Adds `markdown` etc. to results. |
| `answer`, `answerTopN`, `summarizeResults`, `maxCharsPerSource`, `llmApiKey`, `llmProvider`, `llmModel` | | LLM synthesis; need `scrapeOptions`. Not needed by this plugin. |
| Field | Type | Notes |
| ------------------------------------------------------------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------------- |
| `query` | string, required | Max 2000 chars (handler validation). |
| `limit` | int, optional | Default 5 (`default_limit`); clamped server-side to `1..=max_limit` (max 20). |
| `lang` | string | SearXNG language, e.g. `en`, `de`, `auto`; validated as a language tag. |
| `tbs` | enum | `qdr:h`, `qdr:d`, `qdr:w`, `qdr:m`, `qdr:y`. |
| `sources` | array | `web`, `news`, `images`. Presence switches response to grouped form. |
| `categories` | array, max 5 | `github`, `research`, `pdf`, or passthrough string. |
| `scrapeOptions` | object | Scrapes each web result (timeout bounds 1..60000 ms). Adds `markdown` etc. to results. |
| `answer`, `answerTopN`, `summarizeResults`, `maxCharsPerSource`, `llmApiKey`, `llmProvider`, `llmModel` | | LLM synthesis; need `scrapeOptions`. Not needed by this plugin. |
Minimal request is `{"query","limit"}`; `lang`/`tbs` are the only extras worth exposing.
@@ -56,9 +56,19 @@ Envelope `ApiResponse` (camelCase, absent fields omitted): `success`, `data`, `e
Hosted example (flat, per docs):
```json
{"success": true, "data": [
{"url": "https://...", "title": "...", "description": "...", "snippet": "...", "position": 1, "score": 9.5}
]}
{
"success": true,
"data": [
{
"url": "https://...",
"title": "...",
"description": "...",
"snippet": "...",
"position": 1,
"score": 9.5
}
]
}
```
Self-hosted (CLI parser handles `{"success": true, "data": {"results": [ ... ]}}`).
@@ -71,16 +81,16 @@ Conflict note: the docs example and CLI show flat `data[]` for hosted; the struc
JSON body `{"success": false, "error": "<message>", "error_code": "<code>"}` (crates/crw-server/src/error.rs).
| HTTP | error_code | Cause |
|---|---|---|
| 400 | `invalid_request` | bad query/limit/lang |
| 401 | (auth middleware message) | missing/invalid key |
| 429 | `rate_limited` | rate limit |
| 502 | `http_error` | search backend HTTP/JSON failure |
| 504 | `timeout` | search timeout (default 15 s) |
| 503 | `search_disabled` / `search_degraded` | search off or backend unhealthy |
| 422 | `target_unreachable` | backend unreachable |
| 500 | `internal_error` | other |
| HTTP | error_code | Cause |
| ---- | ------------------------------------- | -------------------------------- |
| 400 | `invalid_request` | bad query/limit/lang |
| 401 | (auth middleware message) | missing/invalid key |
| 429 | `rate_limited` | rate limit |
| 502 | `http_error` | search backend HTTP/JSON failure |
| 504 | `timeout` | search timeout (default 15 s) |
| 503 | `search_disabled` / `search_degraded` | search off or backend unhealthy |
| 422 | `target_unreachable` | backend unreachable |
| 500 | `internal_error` | other |
Backend hostnames/credentials are never leaked in errors.
docs/research/opencode-v2-plugin-api.md
+29 -11
View File
@@ -13,15 +13,16 @@ Caveat: pages were read through a fetch-and-summarise tool, so quotes are as ret
## Verdict on the user's sketch
| Sketch element | Verdict | Evidence |
|---|---|---|
| `Plugin.define` from `"@opencode/plugin"` | RIGHT | [S1] `import { Plugin } from "@opencode/plugin"` / `export default Plugin.define({ id: "example", async setup(ctx) {...} })`. V1 package was `@opencode-ai/plugin` [S5]. |
| `ctx.websearch.transform` | RIGHT | [S1] `await ctx.websearch.transform((editor) => { ... })` |
| `editor.add` | RIGHT | [S1] `editor.add({ id: "internal", name: "Internal search", execute: async ({ query }, { signal }) => {...} })` |
| `plugins[].package` / `options` in opencode.json | RIGHT | [S2][S3] key is `"plugins"` (V1 was `"plugin"`, tuples replaced by objects [S5]); object form `{ "package": "...", "options": {...} }` |
| `websearch.provider` | RIGHT (config key) | [S3][S4] `{ "websearch": { "provider": "tavily" } }`; also `"random"`. Built-in ids: exa, firecrawl, parallel, tavily, tinyfish [S4]. |
| Sketch element | Verdict | Evidence |
| ------------------------------------------------ | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `Plugin.define` from `"@opencode/plugin"` | RIGHT | [S1] `import { Plugin } from "@opencode/plugin"` / `export default Plugin.define({ id: "example", async setup(ctx) {...} })`. V1 package was `@opencode-ai/plugin` [S5]. |
| `ctx.websearch.transform` | RIGHT | [S1] `await ctx.websearch.transform((editor) => { ... })` |
| `editor.add` | RIGHT | [S1] `editor.add({ id: "internal", name: "Internal search", execute: async ({ query }, { signal }) => {...} })` |
| `plugins[].package` / `options` in opencode.json | RIGHT | [S2][S3] key is `"plugins"` (V1 was `"plugin"`, tuples replaced by objects [S5]); object form `{ "package": "...", "options": {...} }` |
| `websearch.provider` | RIGHT (config key) | [S3][S4] `{ "websearch": { "provider": "tavily" } }`; also `"random"`. Built-in ids: exa, firecrawl, parallel, tavily, tinyfish [S4]. |
Things the sketch likely missed / must add:
- `id` is required on `Plugin.define` (stable, used for storage scoping/diagnostics) [S5].
- A plugin-added provider is made active with `editor.default.set("<id>")` [S1]; `websearch.provider` in config selects by provider id, so `"provider": "<your-id>"` is the expected pairing (inference from [S1]+[S4]; docs do not state it explicitly -> verify at runtime).
- Options are read as `ctx.options` (not passed as a setup arg) [S1].
@@ -29,47 +30,64 @@ Things the sketch likely missed / must add:
## Sub-questions
### 1. Plugin entry / definition shape
Each module has one default export with a unique id and `setup`:
```ts
import { Plugin } from "@opencode/plugin"
export default Plugin.define({
id: "example",
async setup(ctx) { /* may return a cleanup function */ },
async setup(ctx) {
/* may return a cleanup function */
},
})
```
[S1][S5]. Context: `ctx.app.version`, `ctx.location` (`directory`, optional `workspaceID`, project), `ctx.options`, `ctx.storage` (durable JSON, plugin-scoped), plus domains `ctx.session/permission/shell/tool/websearch/...` [S1][S5]. Return cleanup from `setup` for timers/sockets [S5]. V1 plugins do not run in V2; dual-support is possible via spreading `Plugin.define(...)` and adding a V1 `server()` [S1][S5].
### 2. Registration/loading (local path vs npm)
`"plugins"` array in `opencode.json(c)`; entries are strings or objects [S2]:
npm `"opencode-acme-plugin"`, `"opencode-acme-plugin@1.2.0"`, `"@acme/opencode-plugin"`; local `"./plugins/local"`, `"../shared/plugin.ts"`, `"/abs/plugin.ts"`, `"file:///..."`. Also auto-discovery of `.ts/.js` files and packages in `.opencode/plugins/` or `~/.config/opencode/plugins/` [S2]. CLI: `opencode plugin add|list|check|update|remove` [S2]. Disable via ids: `["*", "-opencode.provider.*", ...]` [S2]. Published package needs `"type":"module"`, `exports["."]` -> entry, dependency `@opencode/plugin` [S1].
### 3. Passing options
```jsonc
{ "plugins": [ { "package": "./plugins/company", "options": { "strict": true } } ] }
{ "plugins": [{ "package": "./plugins/company", "options": { "strict": true } }] }
```
read via `ctx.options.strict` in `setup` [S1][S3]. Options type is untyped key/value (validate yourself).
### 4. Can a plugin provide/replace the websearch provider?
Yes. [S1]:
```ts
await ctx.websearch.transform((editor) => {
editor.add({ id: "internal", name: "Internal search",
editor.add({
id: "internal",
name: "Internal search",
execute: async ({ query }, { signal }) => {
const response = await fetch(`https://search.example.com?q=${query}`, { signal })
return [{ url: "", title: "", content: "", time: {} }]
} })
},
})
editor.default.set("internal")
})
```
Disable: `editor.default.set(false)`; re-evaluate: `await ctx.websearch.reload()` [S1]. The API reference lists add/default/remove operations [S6] (remove not shown in S1 example; exact editor type unverified).
### 5. How `websearch.provider` is selected
Config `{ "websearch": { "provider": "<id>" | "random" } }`; `"random"` picks an available provider; `"websearch": false` disables search entirely [S3][S4]. Provider on HTTP 429 enters cooldown and search retries another provider [S4]. Provider API keys via `/connect` or env (e.g. `TAVILY_API_KEY`) [S4]. For a plugin provider, `editor.default.set(id)` is the plugin-side selection [S1]; interplay/precedence with the config `provider` key is not documented -> verify.
### 6. Required search result shape
From [S1] the `execute` return is an array of `{ url, title, content, time }`; the example uses `time: {}` (object, inner shape undocumented). [S6] lists the same four fields. Unverified: whether `time` fields are optional, exact `time` keys (e.g. published/updated), and whether extra fields are allowed. `execute` receives `({ query }, { signal })`; `signal` is an AbortSignal and must be passed to fetch [S1].
## Open items (need source or runtime check)
- Exact TS types for editor, `time`, and execute args (read anomalyco/opencode source or `@opencode/plugin` typings after install).
- Precedence of config `websearch.provider` vs plugin `editor.default.set`.
- [S6] suggested `POST /api/websearch/query` with `{query, provider?}` -> `{results, provider}`; this came from the summariser and is lower confidence.
docs/spec.md
+25 -16
View File
@@ -8,12 +8,12 @@ An opencode v2 plugin that makes a self-hosted fastCRW server opencode's only we
## Identifiers
| What | Value |
|---|---|
| npm package | `@haylan/opencode-fastcrw` |
| Plugin id (`Plugin.define`) | `fastcrw.websearch` |
| Provider id / display name | `fastcrw` / "fastCRW" |
| Registry | `https://git.arthurerlich.de/api/packages/haylan/npm/` |
| What | Value |
| --------------------------- | ------------------------------------------------------ |
| npm package | `@haylan/opencode-fastcrw` |
| Plugin id (`Plugin.define`) | `fastcrw.websearch` |
| Provider id / display name | `fastcrw` / "fastCRW" |
| Registry | `https://git.arthurerlich.de/api/packages/haylan/npm/` |
## Plugin behaviour
@@ -25,7 +25,11 @@ export default Plugin.define({
async setup(ctx) {
const config = resolveConfig(ctx.options, process.env) // throws on bad values
await ctx.websearch.transform((editor) => {
editor.add({ id: "fastcrw", name: "fastCRW", execute: ({ query }, { signal }) => search(config, query, signal, fetch) })
editor.add({
id: "fastcrw",
name: "fastCRW",
execute: ({ query }, { signal }) => search(config, query, signal, fetch),
})
editor.default.set("fastcrw")
})
},
@@ -38,20 +42,25 @@ fastCRW is the default and only provider; there is no fallback to built-ins.
Precedence: plugin option > env var > default.
| Option | Env var | Default | Rules |
|---|---|---|---|
| `baseURL` | `CRW_API_URL` | `http://localhost:3000` | trim, strip trailing `/`, must be `http(s)` URL |
| `apiKey` | `CRW_API_KEY` | none | optional; Bearer header only when set; never logged or echoed |
| `limit` | none | `8` | clamp to 1..20; invalid falls back to 8, never throws |
| `lang` | none | unset | string; omitted from request when unset |
| `tbs` | none | unset | one of `qdr:h`, `qdr:d`, `qdr:w`, `qdr:m`, `qdr:y`; omitted when unset |
| Option | Env var | Default | Rules |
| --------- | ------------- | ----------------------- | ---------------------------------------------------------------------- |
| `baseURL` | `CRW_API_URL` | `http://localhost:3000` | trim, strip trailing `/`, must be `http(s)` URL |
| `apiKey` | `CRW_API_KEY` | none | optional; Bearer header only when set; never logged or echoed |
| `limit` | none | `8` | clamp to 1..20; invalid falls back to 8, never throws |
| `lang` | none | unset | string; omitted from request when unset |
| `tbs` | none | unset | one of `qdr:h`, `qdr:d`, `qdr:w`, `qdr:m`, `qdr:y`; omitted when unset |
Validation runs in `setup`. A malformed `baseURL`, bad `tbs` or non-string `lang` makes `setup` throw, with a message naming the option and the bad value. Example `opencode.json`:
```jsonc
{
"plugins": [{ "package": "@haylan/opencode-fastcrw", "options": { "baseURL": "http://crw.home:3000", "limit": 8 } }],
"websearch": { "provider": "fastcrw" }
"plugins": [
{
"package": "@haylan/opencode-fastcrw",
"options": { "baseURL": "http://crw.home:3000", "limit": 8 },
},
],
"websearch": { "provider": "fastcrw" },
}
```