- F11 new opt-in module `capture` (skill ckb-capture, capture_url.py): stdlib fetch with readable-text extraction, Wayback only over a failed or thinner live page, --from-file for the agent's own fetch, PDFs saved as files, --media via yt-dlp with consent, untrusted: true marking, dedup at capture time; dedup_inbox.py now compares source_url (D-0017) - F22 tests/run_evals.py (list/prepare/check/validate) and five structural eval cases; reference run for ckb-decide/record - F18 manual section with untested setups for Codex CLI, Cursor, OpenCode and Gemini CLI, plus the Claude Code-only feature table; AGENTS.md tells agents without skill loading to read SKILL.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
66 lines
3.3 KiB
Markdown
66 lines
3.3 KiB
Markdown
---
|
|
type: decision
|
|
tldr: URL and media capture ships as an opt-in `capture` module whose stdlib script writes untrusted-marked snapshots to raw/inbox/ (Wayback only over a failed or thinner live page, agent web tool as fallback, yt-dlp only with consent, dedup at capture); skill behaviour is checked by structural evals with a prepare/check runner and recorded reference runs; multi-agent setup is documented but marked untested.
|
|
status: accepted
|
|
decided_on: 2026-11-02
|
|
decided_by: Michał Kopeć
|
|
affects: .agents/modules/capture/, .agents/skills/ckb-ingest/scripts/dedup_inbox.py, tests/run_evals.py, tests/evals/, AGENTS.md, docs/MANUAL.md
|
|
confidence: 0.7
|
|
quality: 0.8
|
|
last_updated: 2026-11-02
|
|
freshness_window_days: 365
|
|
retention: high
|
|
---
|
|
|
|
# D-0017 — A capture module, skill evals, and untested multi-agent docs
|
|
|
|
**Status:** Accepted · **Decided:** 2026-11-02 · **Deciders:** Michał Kopeć
|
|
|
|
## Context
|
|
|
|
Release 3, batch 3: F11 (URL and media capture), F22 (skill evals) and F18
|
|
(multi-agent setup). Capture fetches content from the open web, so it brings
|
|
untrusted input and a network dependency. Not every KB wants either. Skills
|
|
are prose, and until now the smoke test covered only the scripts.
|
|
|
|
## Decision
|
|
|
|
1. **Capture is a module** (D-0001 pattern), not a base skill, so KBs that
|
|
never clip the web pay no context for it.
|
|
2. **A stdlib script does the fetch** (the user's choice): `urllib`, an
|
|
HTML-to-Markdown extractor that keeps headings, lists and code, and the
|
|
Wayback Machine's closest snapshot. An archived copy replaces the live
|
|
page only when the live fetch failed or yielded less text. When the
|
|
script can't fetch, the agent's own web tool fetches and `--from-file`
|
|
wraps the text the same way.
|
|
3. **Every snapshot is marked untrusted** (`untrusted: true` and a banner)
|
|
and records `source_url`, `fetched_on` and `fetched_via`.
|
|
4. **Dedup at capture time:** the script runs `dedup_inbox.py`, which now also
|
|
compares a capture's `source_url` with archived captures and other inbox
|
|
files, and removes its own duplicate.
|
|
5. **Media only with consent**, through `yt-dlp` when installed, never
|
|
installed by the agent.
|
|
6. **Evals are structural.** `tests/run_evals.py` prepares a scratch KB from
|
|
a fixture, and a person or a CI job runs the agent with the case's prompt.
|
|
`check` then scores files, frontmatter, logs, unchanged pages and lint
|
|
checks. The first cases cover ingest, retrieve (two), decide and retract.
|
|
A case may keep a `reference/` run, which the smoke test uses to check the
|
|
runner both ways.
|
|
7. **Multi-agent docs are marked untested** until someone confirms them. One
|
|
line in `AGENTS.md` tells agents without skill loading to read
|
|
`SKILL.md`. Per-folder rules stay in `## Rules`, which every agent reads.
|
|
|
|
## Alternatives considered
|
|
|
|
- **Capture as a base skill** — context cost for every KB.
|
|
- **Agent web tool only** — less reproducible; rejected by the user.
|
|
- **Evals that grade answer text with a model** — costly and noisy; maybe
|
|
later as an optional layer on top of the structural checks.
|
|
|
|
## Consequences
|
|
|
|
- The live Wayback path could not be exercised while archive.org was
|
|
rate-limiting this machine. The smoke test covers it against a local
|
|
stand-in (`CKB_WAYBACK_API`).
|
|
- Evals need an agent run to mean anything; the release notes should say
|
|
which cases were run.
|