- D-0003: <python> interpreter resolution, per-OS install guide, lint_quick.sh/.ps1 session-start fallback, manual lint checklist, no-script retrieval path, exporters declare Python a hard requirement - Lint check 13 pairs each Crux quote block with the attribution under it, so a multi-source Crux no longer cross-fails; fixture KB gains a second source - Session summaries for 2026-09-29 and 2026-10-01 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
261 lines
13 KiB
Markdown
261 lines
13 KiB
Markdown
---
|
||
name: ckb-ingest
|
||
description: Process raw/inbox/ (or raw/ directly if the inbox is empty) into the structured wiki/: extract typed entities and relationships, synthesize frontmatted pages, cross-link them, then update the index and log. Use for "Ingest" or "Update the wiki". Never for a phrase containing "sync" — that is git, `ckb-sync-changes`.
|
||
---
|
||
|
||
# Ingest skill
|
||
|
||
## Purpose
|
||
|
||
Turn raw, unstructured material in `raw/inbox/` (notes, links, pasted text, scratch
|
||
files) into the local, mutable, structured `wiki/` — the top cascade layer overlaying
|
||
the read-only KBs in `linked/` and `libs/`. This is the primary way the wiki grows.
|
||
|
||
Writes only to `wiki/` and moves processed files within `raw/`. Never touches
|
||
`linked/` or `libs/`, and never touches this repo's git remote; pushing is a separate,
|
||
explicit step (Step 8).
|
||
|
||
## How to run this skill
|
||
|
||
### Step 1 — Process the inbox
|
||
|
||
Scan `raw/inbox/` for new material. After ingesting each item, move it to
|
||
`raw/archive/<YYYY-MM-DD>/`, where the date is today's ingestion date
|
||
(create the dated folder if it doesn't exist yet).
|
||
|
||
If `raw/inbox/` is empty, scan `raw/` directly instead — excluding
|
||
`raw/archive/`, which holds material already processed by a previous
|
||
ingest.
|
||
|
||
### Step 2 — Consult the cascade
|
||
|
||
Before writing anything, check whether the entity already exists: local
|
||
`wiki/` first, then each `linked/<name>/`, then each `libs/<name>/`
|
||
(alphabetical within each layer). The local `wiki/` always wins — upstream
|
||
content is informative context, but it can be overridden locally rather
|
||
than treated as final.
|
||
|
||
### Step 3 — Extract entities
|
||
|
||
Identify typed entities in the source material — people, projects,
|
||
libraries, concepts, systems. Create entity pages at
|
||
`wiki/entities/<entity-name>.md` if they don't already exist. Record typed
|
||
relationships between entities as edge data in `wiki/graph/edges.json`,
|
||
using the closed vocabulary in `wiki/graph/index.md` — `part_of`, `uses`,
|
||
`depends_on`, `produces`, `configures`, `validates`, `implements`, `caused`,
|
||
`contradicts`, `supersedes`. Each verb is defined there by the question it
|
||
answers; if the relationship you have in mind doesn't answer one of those
|
||
questions, it belongs in the page's prose, not in the graph.
|
||
|
||
Also record `has_expertise_in` and `owns` edges when the material actually
|
||
evidences them: `has_expertise_in` when a person demonstrably answers
|
||
questions or explains decisions on a topic, `owns` when they hold
|
||
declared responsibility for a system, area, or decision. These are what
|
||
make "who knows about X" and "who owns X" answerable as a direct graph
|
||
lookup instead of a full-text guess (see `ckb-retrieve` Step 3). Record
|
||
them only from demonstrated evidence — someone being present in a meeting
|
||
or cc'd on a thread is not expertise, and don't infer ownership from job
|
||
title alone. Both are optional, like every other edge type; an absent
|
||
edge is better than a fabricated one.
|
||
|
||
**Decisions are not entity pages.** When the material records a choice that
|
||
was actually made — a call with a rationale, a date, and someone who made it
|
||
— that belongs in `wiki/decisions/` as a numbered decision record, not in
|
||
`wiki/entities/`. Use `ckb-decide`'s format (it owns the field set, the
|
||
numbering, and the two-sided supersession links) and record them in
|
||
chronological order so decision numbers run in the same order as the
|
||
decisions. Two things worth being strict about, because ingest is where they
|
||
go wrong: a proposal that was discussed but not settled is
|
||
`status: proposed`, never `accepted`; and where the source doesn't say who
|
||
decided, `decided_by: unknown` beats attributing it to whoever was loudest in
|
||
the notes. If the material only *references* a decision without recording
|
||
it ("as agreed last quarter, we use Postgres"), that's a query-gap candidate,
|
||
not a decision record — you don't have the rationale or the date.
|
||
|
||
For recurring teams, clients, systems, or initiatives, consider whether a
|
||
plain project scope page under `wiki/projects/<name>.md` would make future
|
||
queries easier to route. A scope page should list when to use it, included
|
||
wiki/entity/source paths, exclusions, and refresh hints. Create one only
|
||
when the source material shows a real repeated scope; don't manufacture
|
||
scopes for one-off facts.
|
||
|
||
### Step 4 — Blast radius: what does this material invalidate?
|
||
|
||
Ingest is additive by habit, which is how a wiki accumulates two pages that
|
||
quietly disagree. Before writing anything, find out what the incoming
|
||
material lands on.
|
||
|
||
For every entity from Step 3 that already has a page, walk
|
||
`wiki/graph/edges.json` **backwards** — every edge whose `to` is that page —
|
||
one hop out. That set, plus the entity pages themselves, is the blast radius:
|
||
the pages whose claims could be affected by what just arrived. Read their
|
||
`tldr`s (and the bodies of any that look genuinely exposed) and sort each one
|
||
into:
|
||
|
||
- **Confirms** — the new material agrees. Note it; corroboration is a reason
|
||
to raise `confidence` in Step 5, not to rewrite anything.
|
||
- **Extends** — the new material adds detail the page doesn't have. Update
|
||
the page in Step 5.
|
||
- **Contradicts** — the new material and the page can't both be right. This
|
||
is the case worth catching: handle it as supersession (see Edge cases), and
|
||
say so explicitly in the report rather than letting the newer page silently
|
||
win.
|
||
- **Untouched** — most of the radius. Drop it and move on.
|
||
|
||
Name the owners too. Where an `owns` or `has_expertise_in` edge points at an
|
||
affected page, that person is who should review the change — surface them in
|
||
the Step 8 report. Skip this step entirely when Step 3 produced no entity
|
||
that already exists; a first ingest into an empty wiki has no radius.
|
||
|
||
### Step 5 — Synthesize pages
|
||
|
||
Convert the core knowledge into clean, modular Markdown files. Every page
|
||
gets frontmatter with:
|
||
- A `tldr:` — one sentence, optimised for LLM reading.
|
||
- A `confidence:` score (0.0–1.0, based on source corroboration).
|
||
- A `quality:` self-score (0.0–1.0).
|
||
- A `last_updated:` timestamp.
|
||
- A `freshness_window_days:` appropriate to the topic.
|
||
- A `retention:` level.
|
||
|
||
`type` is also required (per the page frontmatter schema in
|
||
`CLAUDE.md`/`AGENTS.md` page schema) — set it once, based on the entity/content
|
||
kind (person, project, concept, library, decision, playbook, ...).
|
||
|
||
Every synthesized page ends with the reserved body sections from the page
|
||
schema. `## Sources` and `## Crux` are written here; `## Notes` is created
|
||
empty and never touched again:
|
||
|
||
```markdown
|
||
## Sources
|
||
|
||
- `raw/archive/2026-09-21/kickoff-notes.md` — sha256:3f9a2c1e (checked 2026-09-21)
|
||
- `raw/archive/2026-09-21/access-thread.eml` — sha256:be40d7aa (checked 2026-09-21)
|
||
|
||
## Crux
|
||
|
||
> FDEs need the VDI *and* a Jira account before day one; the VDI request
|
||
> alone takes ten working days.
|
||
— `raw/archive/2026-09-21/kickoff-notes.md`, under "Access"
|
||
|
||
> Access requests go through the platform team, not through IT.
|
||
— `raw/archive/2026-09-21/access-thread.eml`
|
||
|
||
## Notes
|
||
|
||
<!-- Yours. Never rewritten by any skill. -->
|
||
```
|
||
|
||
Compute each fingerprint from the archived file, not from memory:
|
||
|
||
```bash
|
||
sha256sum raw/archive/<date>/<file> | cut -c1-8
|
||
```
|
||
|
||
Record the digest the page was actually built from. That is what lets
|
||
`ckb-lint` tell "old but still accurate" apart from "the source changed
|
||
underneath this page" without asking a model — a calendar window can't
|
||
distinguish those. When a page is built from exactly one source, also set
|
||
`source_fingerprint` and `source_checked` in frontmatter.
|
||
|
||
**`## Crux` is quoted, never paraphrased.** Lift the handful of lines that
|
||
actually carry the claim — the number, the constraint, the condition, the
|
||
commitment — verbatim, and attribute each quote to the source bullet it came
|
||
from: a block of consecutive `>` lines is followed by its own `— \`path\``
|
||
line, and a page may quote several sources this way. `ckb-lint` checks
|
||
each block only against the source named under it. Three to ten lines is the working range; a Crux approaching the length
|
||
of the summary has stopped being evidence and become a second copy of the
|
||
source. The synthesis above it says what the material means; the Crux is what
|
||
it said, so a reader can check the first against the second. This is also
|
||
what lets `ckb-retrieve` ground an answer without opening the archive every
|
||
time.
|
||
|
||
A page with no quotable source — synthesized from conversation, or from
|
||
material too diffuse to quote — simply has no `## Crux`. An empty or
|
||
paraphrased one is worse than none, because it looks like evidence.
|
||
|
||
For long conversations, meeting notes, transcripts, or chat exports, use a
|
||
structured distillation before writing the final page:
|
||
- `Question` or searchable problem statement, when there is one.
|
||
- `Summary` of the thread/note.
|
||
- `Resolution` or `Decision`, if the material contains one.
|
||
- `Systems and code references` mentioned.
|
||
- `People involved` or apparent owners/experts.
|
||
- `High-signal excerpts` for dense technical paragraphs or consecutive
|
||
messages that would be lost in a single summary.
|
||
|
||
"High-signal" needs an acceptance test, or every excerpt looks worth
|
||
keeping and the page becomes a second copy of the transcript. Promote a
|
||
run of text to its own section or linked page only when it clears all
|
||
three:
|
||
- **It carries a rare term.** Something specific enough that a future
|
||
search would use it — a config flag, an error string, a hostname, a
|
||
contract clause, a version number. Check with `rg -c` across `wiki/`:
|
||
if the term already appears on many pages it isn't a distinguishing
|
||
handle, and the excerpt adds no findability the summary doesn't have.
|
||
- **It's substantial.** Roughly 200 characters or more, or a few
|
||
consecutive paragraphs/messages from one author. A one-line "yes, do
|
||
that" is a resolution to fold into `Resolution`, not an excerpt.
|
||
- **Something corroborates it.** It was agreed with, acted on, corrected,
|
||
or referred back to later in the material. An unanswered assertion is a
|
||
claim, not a settled fact — keep it in the summary with that ambiguity
|
||
intact rather than promoting it.
|
||
|
||
Fail any one of the three and the content still belongs in the page, just
|
||
inside `Summary`/`Resolution` rather than as its own retrievable unit.
|
||
|
||
When you do promote an excerpt, carry its parent topic with it — the
|
||
thread question or section heading it sat under. An excerpt that reads
|
||
unambiguously on its own is the entire point; one that needs the
|
||
surrounding transcript to make sense hasn't been extracted, only moved.
|
||
|
||
### Step 6 — Link and cross-reference
|
||
|
||
Use **both** `[[Wikilinks]]` (Obsidian-compatible) and standard
|
||
`[markdown](path.md)` links on every cross-reference, so the wiki works in
|
||
Obsidian, GitHub, and CLI tools alike. Where useful, reference upstream
|
||
files directly at `linked/<name>/...` or `libs/<name>/...`.
|
||
|
||
### Step 7 — Update index and log
|
||
|
||
Add new pages to the routing table in `wiki/index.md` with a **Use when**
|
||
description. If the page lives in a subdirectory, also add it to that
|
||
subdirectory's own `index.md`. Append a log entry to the most specific
|
||
applicable log — the subdirectory's `log.md` if it has one, otherwise the
|
||
root `wiki/log.md` — following the format in Rule B (`CLAUDE.md`/
|
||
`AGENTS.md` Rule B).
|
||
|
||
If this step creates a brand-new `wiki/<topic>/` subdirectory, immediately
|
||
create that subdirectory's `index.md` per the Recursive Index & Log
|
||
Convention.
|
||
|
||
If this step creates or updates a project scope under `wiki/projects/`,
|
||
also update `wiki/projects/index.md`. If ingest closes a previously
|
||
recorded question in `wiki/query-gaps.md`, move that entry from Open to
|
||
Resolved and mention the page or source that now answers it.
|
||
|
||
### Step 8 — Report, then remind to review and sync
|
||
|
||
Report what changed before the reminder: pages created vs. updated, and —
|
||
from Step 4 — which existing pages the new material confirmed, extended, or
|
||
contradicted, naming the owner of each contradicted page. A contradiction
|
||
resolved silently is the one outcome of an ingest a reviewer must not have to
|
||
discover for themselves.
|
||
|
||
Then close with a short reminder — do not sync or push
|
||
anything yourself here, and do not skip this even if the changes look
|
||
small or obviously correct:
|
||
|
||
> "Ingest complete — `wiki/` has been updated (see `wiki/log.md` for the
|
||
> details). Please review the changes, and once you're happy with them,
|
||
> say 'sync changes' to commit and push them to `origin`."
|
||
|
||
If nothing was actually ingested (empty inbox and empty `raw/`, nothing to
|
||
process), skip this reminder — there's nothing to review or sync.
|
||
|
||
## Edge cases
|
||
|
||
Unusual situations — an empty target, a partial run, a conflicting state,
|
||
a missing dependency — are catalogued in
|
||
[`references/edge-cases.md`](references/edge-cases.md). Read that file when a
|
||
run hits something the steps above don't cover; don't load it otherwise.
|