ckb/.agents/skills/ckb-retrieve/SKILL.md
Michał Kopeć 65b1e422b3 Add decision log, scriptify OKF export and lint detection
Decision log (VERSION 1.6.0, kb_schema_version 1.4):
- wiki/decisions/ scaffold — numbered NNNN-slug.md records, own index
  (with status vocabulary) and log
- type: decision adds status/decided_on/decided_by/affects/review_on;
  supersedes/superseded_by carry history and must be set on both sides
- New ckb-decide skill: records decisions and answers what/why/who/when,
  what superseded what, and what is still open. Decision pages are
  append-only — a changed mind is a new superseding decision
- Graph gains decided_by and affects edge types
- ckb-ingest routes decisions found in raw material to this format;
  ckb-retrieve gains the decisions index as a cascade step

Scriptified the mechanical skills:
- ckb-export-okf/scripts/export_okf.py does the whole OKF transform
  (frontmatter remap, link rewriting, index/log regeneration, conformance
  validation); --check validates without writing
- ckb-lint/scripts/lint_report.py does the read-only detection half
  (conformance, freshness, confidence, retention, decisions, orphans,
  graph, index/log, source.yaml); judgment calls stay with the model

Also: removed the duplicate personal quiz skill, fixed stale cbk-quiz
doc paths, gitignored __pycache__.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:50:36 +02:00

17 KiB
Raw Blame History


name: ckb-retrieve description: Governs how a question actually gets answered from the knowledge base. An index entry — a wiki page's tldr, a connector-index document/entity page — is deliberately compressed so lazy-loading stays cheap; that compression also means it can be incomplete, paraphrased loosely, or stale relative to the real source. This skill makes "go verify against the source before answering" a standing step, not an optional last resort: every time a page surfaced by the index looks relevant enough to actually ground part of the answer, follow it to its underlying source material — a wiki page's own ## Sources citations into raw/archive//outputs/, or a connector-index page's resource: pointer back to the live connector item — before treating its content as settled. Use whenever answering a question or researching a topic from wiki/, linked/<name>/, or a connector-backed libs/<name>/ — this is the retrieval workflow routed by CLAUDE.md/AGENTS.md, always in play, not something the user needs to name explicitly. Distinct from ckb-onboard-me (produces a reading-order tour, not an answer to a specific question) and from the write-side skills ckb-ingest/ckb-index-external/ckb-lint (build or repair the index; this skill only ever reads it, plus the sources behind it).

Retrieve (source-verified query) skill

Purpose

The whole point of tldr/lazy-loading (CLAUDE.md/AGENTS.md index-first navigation and skill routing) is that most of the wiki never has to enter context — a one-sentence summary decides whether a page is worth opening at all. That's the right trade for deciding relevance. It's the wrong trade for grounding an answer: a tldr is a compression of whatever the page's author judged important at write time, a connector-index entity page is (per ckb-index-external) deliberately a thin pointer rather than a synthesis, and either can have drifted from the primary material since — a source updated, a nuance dropped, a paraphrase that's subtly wrong.

This skill closes that gap: once a page looks relevant enough to actually use in an answer, don't stop at its tldr or even its full synthesized body — follow it to the source material that page was built from, and answer from there. It's the difference between citing what the index says about the evidence and citing the evidence.

This skill is read-only. It never writes to wiki/, libs/<name>/, or anywhere else — it only reads what already exists (the index, and the source material behind it) to answer the question in front of it. If verification surfaces a real gap or contradiction worth fixing in the wiki, say so and suggest ckb-lint/ckb-ingest rather than editing anything mid-answer.

Trigger phrases

This is the default retrieval path for any question answered from the knowledge base — the user does not need to name this skill. Typical prompts that should route here:

  • A direct question answerable from the wiki ("what do we know about X", "what's the status of Y", "who owns Z").
  • "Look up X" / "check the wiki for X" / "search for X".
  • Mid-conversation moments where CLAUDE.md/AGENTS.md routes a KB question to retrieval — this skill is that workflow's implementation.

Do not use this skill for:

  • Building or refreshing an index in the first place — that's ckb-ingest (raw material → wiki/) or ckb-index-external (connector → libs/<name>/). This skill only ever consumes what those produced.
  • A guided reading order across many pages on a topic — that's ckb-onboard-me. This skill answers one question with verified evidence; it doesn't produce a curriculum.
  • Health-checking the index's own structure (frontmatter, staleness, broken links) — that's ckb-lint. Verification failures found here (Edge cases, below) are worth mentioning to the user as a possible lint finding, but this skill doesn't run the lint checklist itself.

How to run this skill

Step 1 — Read the index

Same cascade order as CLAUDE.md/AGENTS.md, first match wins:

  1. wiki/index.md — match the question against the Use when column.
  2. wiki/projects/index.md — if a project scope matches the question, use that project's listed pages, entity pages, raw/archive sources, connector-backed libs, outputs, and graph areas as the first search boundary. A project scope narrows the first pass only; it never hides the rest of the cascade.
  3. wiki/entities/index.md — match against entity titles/tldr.
  4. wiki/decisions/index.md — for any "why is it like this", "who decided", "when did we choose", or "is that still current" question. If decisions are the subject of the question, hand off to ckb-decide, which owns the decisions-scoped path including supersession chains; pull decision pages in here as one source among many when they're only part of a wider answer. Either way, a superseded decision is history, not noise — say so rather than silently answering from the current one.
  5. If nothing local matches: each linked/<name>/ index (alphabetical), then each connector-backed libs/<name>/entities/index.md — for a connector-backed lib this means its generated index (both the Documents and the Entities & Processes sections ckb-index-external produces), never the live source directly. If it isn't built yet, suggest "index external sources" rather than querying the live connector ad hoc from inside this skill.

Step 2 — Shortlist every page that looks relevant

Read the tldr of every page the index match surfaced — don't stop at the first plausible hit. A question is often best answered by triangulating two or three pages (e.g. a concept page plus the specific person/decision page that qualifies it), and a page whose tldr looks only tangential can still be hiding the exact fact needed in its body or its Sources. Keep the shortlist to what's plausibly relevant; this isn't "open everything," it's "don't stop at one."

If index/TLDR matching is not enough, run a local hybrid pass before giving up: use rg for exact tokens across wiki/, raw/archive/, outputs/, raw/inbox/, and readable upstream indexes; combine those hits with project-scope matches, entity/title/TLDR matches, graph proximity, freshness, confidence, and quality. Prefer exact text matches for error strings, commands, flags, filenames, hostnames, IDs, and other literals; prefer entity/semantic matches for paraphrased questions.

Sweep raw/inbox/ even though nothing there has been ingested yet. Material dropped an hour ago can already hold the answer, and finding it there is also the clearest possible signal that an ckb-ingest run is overdue — mention that. Say plainly when an answer rests on un-ingested inbox material rather than on a synthesized page.

Keep each signal's hits as its own ranked list rather than merging them by eye as you go. Step 4 needs the separate orderings.

Step 3 — Walk the graph for anything the shortlist missed

If a shortlisted page has edges in wiki/graph/edges.json (or a connector's own graph/edges.json), follow depends_on/uses/caused edges one hop out to catch a connected page the index text match alone wouldn't have surfaced.

For "who knows about X" and "who owns X" questions — both advertised in the trigger phrases above — the graph is the primary lookup rather than a fallback. Read the has_expertise_in and owns edges pointing at the topic entity and answer from the people or teams on the other end, ordered by how many distinct sources evidence each edge. Where no such edge exists yet, fall back to authorship evidence: who the ## Sources material actually shows answering questions on that topic. Say which of the two grounded the answer, since an inferred expert is a weaker claim than a recorded one.

Step 4 — Fuse the signals, dedupe, then rerank

Signals disagree, and none of them is trustworthy alone: an exact rg hit can sit in a page that answers a different question, and a confident TLDR match can be a loose paraphrase of something stale. Don't settle that by picking a favourite signal — fuse the ranked lists from Step 2.

Fuse. For every candidate, sum weight / (k + rank) across each list it appears in, where rank is its 1-based position in that list. Use k = 10 and a default weight of 1.0. A page ranked third in three different lists beats one ranked first in a single list; consensus is the whole point.

k = 10 is deliberately smaller than the k = 60 rank fusion is usually quoted with. 60 is tuned for retrievers returning hundreds of candidates, and against the dozen-or-so a local wiki produces it flattens every score into near-identical values. Raise a list's weight when the question warrants it — for a pasted error string, command, flag, or ID, weight the exact-match list around 2.0, because no amount of title/TLDR similarity should outrank a literal match on the token the user actually pasted.

Dedupe. Collapse candidates carrying the same claim into one entry before ranking further. Step 6 actively manufactures these: a wiki/ page, the raw/archive/ file it cites, and a connector-index page whose resource: points back at that same wiki page are three hits for one fact. Keep whichever sits closest to the primary material and record the others as corroboration, not as independent evidence. Three views of one claim are not three sources.

Rerank. Score each surviving candidate 010 on how well it answers the literal question asked, not on how well it matches the query's vocabulary, then keep the best handful and drop the rest. This is the same agent making a deliberate second pass, not a separate model. The point is that relevance judgment happens explicitly, over the shortlist, in one place — rather than being folded silently into how the final answer gets drafted.

Step 5 — Expand local context and build an evidence packet

For every result that survived Step 4, keep a small evidence packet with:

  • source path or connector resource
  • matched claim or short excerpt
  • source date or last_updated
  • freshness/confidence/quality signals, when available
  • project-scope or graph relationship hints, when relevant
  • which signals it was fused from, its rerank score, and anything it absorbed during dedupe — this is what Step 7 draws caveats from

When a match is a section, heading, or snippet inside a larger Markdown file, include nearby headings/paragraphs before deciding what it means. Avoid answering from an isolated fragment when the neighboring context changes the interpretation.

Step 6 — Follow every page on the shortlist to its source before answering from it

This is the step this skill exists to enforce. For each page on the shortlist that will actually ground part of the answer:

  • A wiki/ page — every page synthesized via ckb-ingest carries a ## Sources section citing the exact raw/archive/<date>/... or outputs/... file(s) it was built from. Open the cited file(s) — or the specific section of a large one — and confirm the wiki's claim matches what the primary material actually says. This also often surfaces adjacent detail the synthesis compressed away that's directly useful for the current question.
  • A connector-index page in libs/<name>/ (a document page, or an entity/process page from the ckb-index-external extraction) — read its resource: field. If the connector is authorized this session (per ToolSearch/MCP auth state), re-fetch the live item for current, complete content rather than trusting the cached tldr — connector pages use a deliberately short freshness_window_days (30, vs. a typical wiki page's 60-90) precisely because external sources drift without notice. If the connector isn't authorized, or the item is a calendar acceptance / metadata-only page with nothing more to fetch, say plainly that the answer relies on the cached index rather than a live re-check, so the user can weigh that.
  • A linked/<name>/ page — this is already full mirrored content, not a summary; reading the page itself already is reading the source. No extra fetch needed unless that page itself cites something further outside the mirror.

Target the specific claim, not the whole file — a large transcript or deck doesn't need a full read every time, just enough (search for the name/topic, read the surrounding context) to confirm the point actually being used.

Step 7 — Reconcile and answer

If the source confirms the index, answer normally — but note what grounded it (e.g. "per the kickoff transcript cited in gt-fde-access-requirements.md") rather than presenting the answer as if sourced from the tldr alone.

If the source contradicts, extends, or is more precise than what the index said, prefer the source for the answer and say so explicitly — this is exactly the kind of drift ckb-lint's freshness/confidence checks exist to eventually catch, so mention it's worth a lint pass if the gap looks like more than a one-off, but don't rewrite the wiki page yourself mid-answer unless the user asks for that separately.

State caveats in the answer itself, not only in the metadata you read to build it. When a page grounding the answer is past its freshness_window_days, carries a low confidence or quality, rests on un-ingested raw/inbox/ material, or was checked against a cached connector index rather than a live re-fetch, say so in a short clause next to the claim it qualifies. Surface a conflict between two live pages the same way, even when neither is marked superseded_by yet. The metadata already exists and Step 4 already put it in front of you; the failure mode is answering confidently from a stale or contested page without passing that on, which leaves the reader no way to weigh it.

If no page anywhere is even plausibly relevant, say so — don't verify against a source that has nothing to do with the question just to appear thorough, and don't fabricate an answer either. For durable missing knowledge, add or propose a short wiki/query-gaps.md entry with the question, date, search areas tried, and the smallest missing source/page that would close the gap. If you edit wiki/query-gaps.md, update wiki/log.md immediately.

Edge cases

  • Only one signal returned anything — fusion is a no-op, so don't run the arithmetic for show. The rerank still applies (it's the check that the one list actually answers the question rather than merely matching its wording), and dedupe still applies as soon as Step 6 pulls in a page's own cited source.
  • A cited source file no longer exists (moved, renamed, or cleaned up) — say so plainly rather than silently falling back to the tldr as if it had been verified. Flag it as a likely ckb-lint finding (a broken Sources reference) rather than treating it as a dead end.
  • Connector item unreachable or connector not authorized this session — state clearly that the answer is based on the cached connector index, not a live re-check, and name which connector would need authorizing to go further.
  • Question is low-stakes and the matched page is high-confidence, recently updated, and well within its freshness_window_days — still open the source at least once to ground the answer; skip only a second redundant read if the same source has already been opened earlier in the same retrieval pass for another part of the same question.
  • Many pages are relevant and reading every cited source in full would be excessive — prioritize the sources that actually carry the specific fact the question needs, not every source any shortlisted page has ever cited. Say what was skipped rather than silently narrowing without a note.
  • The "source" is itself a generated artifact (e.g. an outputs/emails/*.md draft cited as a wiki page's Source) rather than raw external material — that's still the source relative to the wiki page; no further hop beyond it is required.
  • A connector-index entity/process page's resource: points at a full wiki/entities/ page (per ckb-index-external's thin-page convention) — treat the wiki page as the real source to verify against; the connector page is just the pointer that got you there.
  • Nothing in the index is even plausibly relevant — say the knowledge base has nothing on this yet, and suggest ckb-ingest (for new raw material) or ckb-index-external (for a connector-backed source) if that seems like the actual gap. For durable gaps, record or propose a wiki/query-gaps.md entry. Don't stretch a weak match into an answer just to have one.

Licensed under the Apache License, Version 2.0 — see LICENSE at the repository root, or http://www.apache.org/licenses/LICENSE-2.0.