Improvement memory: attempts, expected outcomes, lesson context (D-0023)

Schema 1.8, after Google's Self-Improvement Memory paper (arXiv 2611.18856).
- Error book: one entry per recurring problem with every attempt, its
  expected and observed result and context; aliases on merge (ckb-lint §9).
- Decisions: expected_outcome, outcome, outcome_checked, ## Tried,
  append-only ## Outcome, an outcome-review procedure (ckb-decide); lint
  validates outcomes and closes a due review once outcome_checked >= review_on.
- Lessons: Context line, upstream lessons from linked/ in the session
  brief, adopt and merge rules (ckb-profile, session_start.py); ckb-init
  Step 8b carries lessons and problems into a new KB; ckb-upgrade b5.
- Deferred: a self-improvement loop over the template. Rejected: loading
  the whole record into context.
No VERSION change or changelog row: made on the merge from dev (D-0022).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Michał Kopeć 2026-11-18 08:17:09 +01:00
parent 9c0a888e98
commit 03ff809440
20 changed files with 654 additions and 62 deletions

View file

@ -10,6 +10,9 @@ description: Record a numbered, append-only decision record in wiki/decisions/,
A decision is never rewritten to reflect a later change of mind: a new one
supersedes it and both stay on the record. Recording wires supersession in
both directions; look-up is scoped to `wiki/decisions/`, not the whole wiki.
It also keeps what the decision was expected to change and, at review,
what actually happened, so a later reader can tell a decision that worked
from one that merely stayed in force.
Writes only `wiki/decisions/`, its `index.md` and `log.md`,
`wiki/graph/edges.json`, and a pointer line in `wiki/log.md`.
@ -28,6 +31,7 @@ decided_on: 2026-09-01
decided_by: Alice Smith, Bob Jones
affects: /wiki/entities/billing-service.md, /wiki/projects/index.md
review_on: 2027-03-01
expected_outcome: Monthly reporting queries run in the billing store, with no second analytics database.
supersedes: /wiki/decisions/0003-use-mysql.md
confidence: 0.9
quality: 0.8
@ -66,6 +70,18 @@ What else was on the table and why each was not chosen. A decision record
without this is much less useful on re-reading: it's the part that stops
the same option being re-proposed every six months.
## Tried
What was actually built, run or tested on the way here and dropped, with
what was observed. Not the same as *Alternatives considered*: those were
weighed, these were tried. A pilot that read MySQL replicas into a
reporting cache ran for two weeks (2026-08) and missed its nightly window
three times.
## Outcome
*(Appended at review, never before. See "Reviewing an outcome".)*
## Supersession
Supersedes [[D-0003]] / [D-0003](/wiki/decisions/0003-use-mysql.md) — MySQL
@ -111,7 +127,10 @@ the append-only rule exists to prevent.
| `decided_on` | yes for `accepted`/`rejected`/`reversed` | `YYYY-MM-DD`, the date the call was made. Distinct from `last_updated`, which is when the *page* last changed. A `proposed` decision may have no `decided_on` yet. |
| `decided_by` | yes when known | comma-separated names, kept flat and plain so tooling can read it. The body's **Deciders** line carries the dual-links to entity pages. If genuinely unknown, write `unknown` rather than omitting the field — "we don't know who decided this" is itself worth recording. |
| `affects` | no | comma-separated project-root-absolute wiki paths this decision constrains. Cheap way to answer "what decisions touch X" without walking the graph. |
| `review_on` | no | `YYYY-MM-DD` to revisit. `ckb-lint` reports these once the date passes. |
| `review_on` | no | `YYYY-MM-DD` to revisit. `ckb-lint` reports these once the date passes, until an `outcome_checked` on or after it records the review. |
| `expected_outcome` | no, preferred with `review_on` | One line: what this decision should change, stated so a later reader can check it ("p95 checkout latency under 300 ms"; not "better performance"). Written **when the decision is recorded**, before anyone knows the result. One added later gets `(added YYYY-MM-DD)` at its end, so nobody mistakes hindsight for a prediction. |
| `outcome` | set at review | `met` / `partly` / `missed` / `unclear`: how what happened compares with `expected_outcome`. |
| `outcome_checked` | set at review | `YYYY-MM-DD` of the latest outcome review. |
| `supersedes` / `superseded_by` | when relevant | project-root-absolute path to the other decision. **Always set both sides** (see below). |
| `confidence`, `quality`, `last_updated`, `freshness_window_days`, `retention` | as usual | standard page schema. Decisions default to `freshness_window_days: 365` and `retention: high` — a decision record doesn't rot the way a status page does, and it should survive a retention sweep. |
@ -127,6 +146,13 @@ Then check what's missing against this list, in priority order:
3. **When** — a date. "Today" is fine; "last Tuesday's architecture review" is fine, resolve it to a date.
4. **Why** — the rationale, and the alternatives that lost.
5. **Whether it changes an existing decision** — see Step 2.
6. **What it should change** — the `expected_outcome`, and a `review_on`
date by which that should be visible. Offer one from the rationale if
the user gave none ("So by March, reporting runs without a second store?
Shall I put that down to check?"). Never invent it: no answer means no
field.
7. **Anything already tried** — a pilot, a spike, a reverted change. That
goes under `## Tried` with what was observed.
Ask for the missing ones in a **single** `AskUserQuestion` round rather than
an interview — this is a recording task, not a discovery interview. If the
@ -158,6 +184,14 @@ same subject. If one exists and is still `accepted`:
Both directions must be set. A one-sided supersession link is a lint finding,
and it breaks the "what changed this?" lookup in the other direction.
**A direction already tried or rejected.** When the new decision picks
something listed under an earlier decision's `## Tried` or `## Alternatives
considered`, or reverses a `rejected` decision, say so and ask what is new
since then: a changed constraint, new evidence, a different model or
setting. Put that answer in the new decision's `## Context`. Coming back to a
direction is fine; coming back without knowing it was tried before is the
mistake this check prevents.
### Step 3 — Allocate the number and write the page
The next number is the highest existing `NNNN` in `wiki/decisions/` plus one,
@ -201,6 +235,33 @@ updated), which deciders had no entity page, and the standard reminder that
this is on disk but not committed — "say 'sync changes' when you want it
pushed."
## How to run this skill — reviewing an outcome
Triggered by "review D-0007", "did D-0007 work", or a lint finding that a
`review_on` with an `expected_outcome` has passed.
1. Read the page: `expected_outcome`, `## Rationale`, `## Tried`.
2. Ask the user what actually happened, or check it against the wiki where
the evidence is recorded there. Cite where the answer came from.
3. Append a dated entry to `## Outcome` (create the section above
`## Supersession` / `## Sources` if missing). Never edit an earlier one:
```markdown
### 2027-03-04 — partly
- **Expected:** Monthly reporting queries run in the billing store, with no second analytics database.
- **Observed:** Monthly reports run in Postgres; the yearly audit export still needs the old warehouse.
- **Evidence:** /wiki/projects/billing.md, ops review 2027-03-02
- **Checked by:** Ana Reyes
```
4. Set `outcome`, `outcome_checked` and `last_updated`. If another check is
worth making, set a new `review_on` after `outcome_checked`. Otherwise
leave `review_on` as it is: the outcome check closes it.
5. When the outcome says the decision should change, that is a new decision
that supersedes this one (recording, Step 2), and this outcome entry is
its context. Offer it; do not record it unasked.
6. Log it in `wiki/decisions/log.md` (action `UPDATE (outcome review)`).
## How to run this skill — looking up
### Step 1 — Start at the decisions index
@ -222,7 +283,9 @@ ones.
and before that MySQL, changed in September because of reporting" is the
useful answer; "we use Postgres" is the impoverished one. Follow
`supersedes`/`superseded_by` chains in both directions and present the
history in order.
history in order. Include `outcome` where one is recorded: "in force, and
reviewed in March as having partly met its aim" is a different answer from
"in force".
### Step 3 — Verify against the source before answering
@ -255,10 +318,14 @@ a `wiki/query-gaps.md` entry (`ckb-retrieve` owns that file).
## Rule: decisions are append-only
The only edits this skill makes to an *existing* decision page are:
`status`, `superseded_by`, `last_updated`, and — when the user is explicitly
`status`, `superseded_by`, `last_updated`, the outcome fields
(`outcome`, `outcome_checked`, a later `review_on`), a new dated entry
appended to `## Outcome`, and — when the user is explicitly
correcting a recording error rather than changing their mind — the factual
metadata fields. Context, Decision, Rationale, Consequences, and Alternatives
are never rewritten to match a later view.
metadata fields. Context, Decision, Rationale, Consequences, Alternatives, Tried,
`expected_outcome`, and earlier Outcome entries are never rewritten to match
a later view. A wrong prediction is the most useful thing on the page; the
reason to write it down first is so that it can turn out wrong.
When the user says "actually we changed our mind about D-0003", that is a new
decision superseding D-0003, not an edit to D-0003. Say so and record it that

View file

@ -315,8 +315,9 @@ generic structure:
generic - directory tree, cascade priority, frontmatter summary, no
project content). Set `last_updated` to today.
- **`wiki/log.md`** - header and explanation only, no entries.
- **`wiki/error-book.md`** - copy verbatim (already generic: empty table
+ placeholder note). Set `last_updated` to today.
- **`wiki/error-book.md`** - copy verbatim (already generic: the entry
format in a comment + a placeholder note, no problems). Set
`last_updated` to today.
- **`wiki/entities/index.md`** - header + placeholder note, no entries.
- **`wiki/graph/index.md`** - header + pointer to `edges.json`, the edge
vocabulary table and its conventions **verbatim** (`ckb-ingest`,
@ -342,6 +343,37 @@ If the template source's `wiki/` holds scaffold files beyond this list,
carry them over the same way - empty, structure only. The list above is
what the schema requires, not a cap.
#### Step 8b - Offer to carry experience from this KB (only when run inside one)
A new KB starts with no lessons and an empty error book, so it can repeat
mistakes this KB has already learned from. When this skill runs inside a
populated KB whose `wiki/lessons.md` or `wiki/error-book.md` has entries, ask
once:
> "This KB has N lessons and M error-book problems. Carry any into the new
> KB? (all / pick / none)"
Default to none, and never carry anything unasked. On *pick*, list each
lesson's heading and **Context** and each error-book entry's heading and
status, and take the user's choice. For each one carried:
- **Lesson** → the new KB's `wiki/lessons.md` (create it as `ckb-profile`
does), unchanged except for an added
`- **Carried from:** <source KB name>, recorded there YYYY-MM-DD` line.
Skip struck-through lessons.
- **Error-book problem** → the new KB's `wiki/error-book.md` with all its
attempt rows, the heading kept, and a `- **Carried from:** <source KB
name>, YYYY-MM-DD` line under **Opened**. Its status becomes `watching`
whatever it was: nothing has been checked in the new setting yet.
- Set `review: needs_update` on both files so someone in the new KB checks
what was carried, and add one entry to the new `wiki/log.md` naming the
source KB and the count.
Profiles (`workload/profiles/`) and personal lessons are never carried: they
belong to people, not to a KB. A lesson whose **Applies when** names this
KB's own projects or sources rarely makes sense elsewhere. Point that out
when the user picks one.
Do not carry over any entity pages, graph edges, log entries, or overview
content specific to this project - the whole point is an empty KB with the
same shape.
@ -443,6 +475,7 @@ Tell the user:
- The `VERSION` the new KB starts on, and what was decided for `LICENSE`'s
copyright line.
- Whether a git repo was initialized in the target, or why not.
- What experience was carried (Step 8b): counts, or "none".
- Next step: "Drop material into `raw/inbox/` and say 'Ingest' to populate the wiki for the first time."
## Edge cases

View file

@ -117,11 +117,15 @@ from the vocabulary, a `decided_on` date where the status implies one, a
`decided_by` (`unknown` counts — an omitted field doesn't), `affects` targets
and supersession links that resolve, supersession set on **both** sides,
`status: superseded`/`reversed` matched by a `superseded_by`, unique
four-digit decision numbers, and any `review_on` date that has passed. Fix
those directly — they're mechanical. A `review_on` that has come due is not a
defect, though: report it so the user can decide whether to revisit, and never
change a decision's substance yourself (`ckb-decide` → decisions are
append-only).
four-digit decision numbers, an `outcome` from `met`/`partly`/`missed`/`unclear`
with a dated `outcome_checked`, and any `review_on` date that has passed
with no `outcome_checked` on or after it. Fix the structural ones
directly — they're mechanical. A `review_on` that has come due is not a
defect, though: report it so the user can decide whether to revisit. When
the page has an `expected_outcome`, the finding says so: that review is
"did it do what we expected?", answered through `ckb-decide`'s outcome
review. Never change a decision's substance yourself (`ckb-decide` →
decisions are append-only).
The *semantic* half stays yours: when two pages appear to cover the same
entity or the same choice, check for contradictions. If one is clearly
@ -159,9 +163,40 @@ duplicate log entries directly — these are the safest auto-fixes on the list.
Record any systemic issue found above (a repeated broken pattern, a
recurring format mismatch, the same kind of orphan appearing again) in
`wiki/error-book.md` with its root cause, the fix applied, and the derived
constraint that should prevent recurrence. A one-off, isolated issue
doesn't need an Error Book entry — this is for patterns, not incidents.
`wiki/error-book.md`. A one-off, isolated issue doesn't need an entry —
this is for patterns, not incidents. The file's comment block holds the
format; the rules:
1. **Problem first, then attempt.** An entry (`## E<n> — <root cause>`) is
a hypothesis about *why*, and holds every attempt made against it. Read
the open entries before writing. When one already explains the finding,
append an attempt row to it. Open a new entry only when none does.
2. **Write the expectation before the result.** Each attempt states what
it should stop, in terms the next lint run can confirm ("no more
orphans under `wiki/projects/`"), with `Observed: pending`.
3. **Close the loop on later runs.** On every lint run, check each
`pending` attempt against this run's findings and fill in `held`,
`recurred` or `partly`, with the date. When the problem recurs, the old
row stays as written and a new attempt is added. A tested fix that did
not hold is evidence, not clutter. Try the same fix again only when
something has changed, and say what changed in the new row.
4. **Context on every attempt**: the agent/model and the template
`VERSION` it ran under. When the model or template has changed since an
attempt, treat its result as something to recheck, not a settled fact.
5. **Status**: `open` while it recurs; `watching` once an attempt held but
fewer than two later runs have confirmed it; `resolved` after that. Add a
`Constraint:` line once one is known, and propose it for `AGENTS.md` if
it should bind every session (never edit `AGENTS.md` from here).
6. **Merging**: two entries with the same root cause are merged into the
older one. Move the attempt rows across, marked `(from E4)`, and leave
the newer heading as an alias, `## E4 → E2 — merged YYYY-MM-DD`, so
links and log entries that name E4 still resolve.
An error book from before schema 1.8 (a single table) stays as it is under
an `## Earlier entries` heading. New entries use the format above; old rows
are moved into an entry only when a new attempt joins them.
Every change to the file is a `wiki/` change, logged under Rule B.
### 10 — External source config check

View file

@ -51,6 +51,7 @@ LIBS = REPO_ROOT / "libs"
RESERVED = {"index.md", "log.md"}
DEFAULT_REFRESH_DAYS = 30
DECISION_STATUSES = {"proposed", "accepted", "rejected", "superseded", "reversed"}
DECISION_OUTCOMES = {"met", "partly", "missed", "unclear"}
# --------------------------------------------------------------------------
@ -319,15 +320,32 @@ def check_decisions(docs, root, today):
if not doc.fm.get("decided_by", "").strip():
findings.append(f"{rel}: no `decided_by` — record `unknown` rather than omitting it")
# Expected vs observed outcome (schema 1.8): a review is done once
# `outcome_checked` is on or after `review_on`.
outcome = doc.fm.get("outcome", "").strip().lower()
if outcome and outcome not in DECISION_OUTCOMES:
findings.append(
f"{rel}: `outcome: {outcome}` is not one of {', '.join(sorted(DECISION_OUTCOMES))}"
)
checked_raw = doc.fm.get("outcome_checked")
checked = as_date(checked_raw) if checked_raw else None
if checked_raw and checked is None:
findings.append(f"{rel}: `outcome_checked` is not a YYYY-MM-DD date: {checked_raw!r}")
if checked and not outcome:
findings.append(f"{rel}: `outcome_checked` set but no `outcome`")
expected = doc.fm.get("expected_outcome", "").strip()
review = doc.fm.get("review_on")
if review:
review_date = as_date(review)
if review_date is None:
findings.append(f"{rel}: `review_on` is not a YYYY-MM-DD date: {review!r}")
elif review_date <= today:
elif review_date <= today and not (checked and checked >= review_date):
findings.append(
f"{rel}: due for review — `review_on: {review_date.isoformat()}` "
f"passed {(today - review_date).days}d ago"
+ ("; check `expected_outcome` against what happened and record `outcome`"
if expected else "")
)
# Supersession links must resolve and must be reciprocal.

View file

@ -18,6 +18,7 @@ What it prints, one line each (plus the tail of the latest workload note):
threads: N open ... unticked `- [ ]` under "Open threads" in workload/ notes; only when any
profile: workload/profiles/<slug>.md the session identity's own profile, printed; only when it exists
lessons: N new in wiki/lessons.md KB lessons dated after your previous workload note; only when any
upstream lessons: N in linked/<name>/... the same for each linked KB's lessons, read-only, ranked below local ones
workload: <file> (last block follows) latest workload/ note by filename date
Exit 0 unless the repo root is not a KB. Python 3.8+, standard library only.
@ -269,23 +270,46 @@ def profile_lines(root, slug):
"project rules leave room"] + [" " + l if l else "" for l in body]
def lessons_lines(root, slug, today):
"""KB lessons dated after this person's previous workload note (all of them
on a first session), so a lesson someone else recorded is not missed."""
path = root / "wiki" / "lessons.md"
def read_lessons(path):
"""(date, rule) for each lesson heading, skipping struck-through ones
(`~~rule~~ — dropped/merged …`): those stay on the record but are not
advice any more."""
if not path.is_file():
return []
lessons = LESSON_RE.findall(path.read_text(encoding="utf-8-sig", errors="replace"))
prev = None
if slug and (root / "workload").is_dir():
dates = [f.name[:10] for f in (root / "workload").glob(f"*_{slug}_summary.md") if f.name[:10] < today.isoformat()]
prev = max(dates) if dates else None
new = [(d, t) for d, t in lessons if prev is None or d > prev]
if not new:
return []
found = LESSON_RE.findall(path.read_text(encoding="utf-8-sig", errors="replace"))
return [(d, t) for d, t in found if not t.startswith("~~")]
def previous_note(root, slug, today):
"""Date of this person's latest workload note before today, or None."""
if not slug or not (root / "workload").is_dir():
return None
dates = [f.name[:10] for f in (root / "workload").glob(f"*_{slug}_summary.md") if f.name[:10] < today.isoformat()]
return max(dates) if dates else None
def lessons_lines(root, slug, today):
"""KB lessons dated after this person's previous workload note (all of them
on a first session), so a lesson someone else recorded is not missed; then
the same for each linked KB's lessons, marked upstream (prior experience
from another setting, ranked below every local lesson)."""
prev = previous_note(root, slug, today)
since = f"since your note of {prev}" if prev else "— first session here"
out = [f"lessons: {len(new)} in wiki/lessons.md {since} -> read the ones that bear on this session"]
out += [f" - {d} — {t}" for d, t in new[-THREADS_SHOWN:]]
out = []
new = [(d, t) for d, t in read_lessons(root / "wiki" / "lessons.md") if prev is None or d > prev]
if new:
out.append(f"lessons: {len(new)} in wiki/lessons.md {since} -> read the ones that bear on this session")
out += [f" - {d} — {t}" for d, t in new[-THREADS_SHOWN:]]
linked = root / "linked"
if linked.is_dir():
for kb in sorted(p for p in linked.iterdir() if p.is_dir()):
up = [(d, t) for d, t in read_lessons(kb / "wiki" / "lessons.md") if prev is None or d > prev]
if not up:
continue
rel = f"linked/{kb.name}/wiki/lessons.md"
out.append(f"upstream lessons: {len(up)} in {rel} {since} -> another KB's experience; "
"local lessons win; check its Context before relying on one")
out += [f" - {d} — {t}" for d, t in up[-THREADS_SHOWN:]]
return out

View file

@ -73,9 +73,17 @@ most, offer once: "Want me to keep that as a lesson?"
- **Why:** <what went wrong or what the person said>
- **Applies when:** <the situations it covers>
- **Scope:** person | KB
- **Context:** <agent/model>, template <VERSION>[, <the setting: project, kind of source, task>]
- **Recorded by:** Name
```
**Context** records the situation the lesson was learned in: the agent and
model the session ran under (`unknown` if you cannot tell), the template
`VERSION`, and, when it matters, the project or kind of material. A lesson
learned while one model kept skipping a step may not hold for the next
model. Without this line there is no way to tell which lessons to
recheck after a change.
Ask which scope when it is not obvious:
- **Person** — how this person likes to work. Append to `## Lessons` in
@ -93,6 +101,34 @@ Suggest that once; do not edit `AGENTS.md` from here.
The session-start brief lists KB lessons dated after the person's previous
workload note. Read the ones that bear on the task.
**How to apply a lesson.** A lesson is prior experience, not a rule of the
same rank as `AGENTS.md`:
- Check its **Context** against the current session. The same model,
template major version and kind of task means it applies as written. A
different one means it is a strong hint: follow it unless the current
evidence says otherwise, and if it turns out not to hold, say so and offer
to strike it through or narrow its **Applies when**.
- A lesson with no **Context** line (recorded before schema 1.8) is read
as context unknown.
- When two lessons conflict, the one whose context is closer to the
current session wins. Mention the conflict once and offer to merge or
narrow them.
**Upstream lessons.** The brief also lists lessons from `linked/<name>/wiki/lessons.md`,
marked `upstream`. They are read-only (Rule A) and rank below every local
lesson. They describe another KB's setting, so apply them as above with the
context treated as different. To make one local, on request ("adopt that
lesson"), copy it into `wiki/lessons.md` with today's date and one extra
line, `- **Carried from:** linked/<name>, recorded there YYYY-MM-DD`, then
set `review: needs_update` and log it like any KB lesson. `ckb-init`
carries lessons into a new KB the same way.
**Merging**: when two lessons say the same thing, keep the older one, add
`- **Also recorded:** YYYY-MM-DD (merged)` to it, and strike through the
newer one with `~~rule~~ — merged into "<older rule>" YYYY-MM-DD`. Both
stay, so a link or log entry naming either still finds it.
**Forgetting**: a lesson is never deleted silently. Strike it through
(`~~rule~~ — dropped YYYY-MM-DD: why`) so the record survives. A KB lesson
change is logged under Rule B.

View file

@ -429,6 +429,22 @@ template's scaffold if it is missing, and add its routing row to
`wiki/index.md`. Topic pages themselves are never generated by a migration;
`ckb-consolidate` proposes and builds them on request.
**b5. Schema 1.7 → 1.8: outcomes, attempts, lesson context.** Every 1.8
addition is optional (D-0023), so a 1.7 page is already a valid 1.8 page.
- **`wiki/error-book.md`** — if it holds the old single table with rows,
put the table, unchanged, under an `## Earlier entries` heading. Then add
the 1.8 entry-format comment above it from the template's scaffold, and
update the `tldr`. Never convert old rows into problem entries: an
attempt's expected result was never written down, and making one up now
would be hindsight presented as a prediction. An empty table is replaced
by the scaffold.
- **Decisions** — do not backfill `expected_outcome`. Offer once: "N
decisions have a `review_on` and no expected outcome. Want to add one to
each? It will be marked `(added <today>)`." Apply only on a yes, one
decision at a time with the user's wording.
- **Lessons** — do not backfill **Context**; a lesson without one is read
as context unknown (`ckb-profile`).
**c. Bump `wiki/index.md`'s `kb_schema_version`** to the template's
current value, once every page has been checked.

View file

@ -187,12 +187,14 @@ evidence: follow it to its members before grounding an answer.
Pages with `type: decision` live in `wiki/decisions/` as `NNNN-slug.md` and
add `status` (`proposed`/`accepted`/`rejected`/`superseded`/`reversed`),
`decided_on`, `decided_by`, and optionally `affects` and `review_on`. They are
`decided_on`, `decided_by`, and optionally `affects`, `review_on` and
`expected_outcome` (what the decision should change, written when it is made,
checked at review into `outcome`/`outcome_checked`). They are
append-only: never rewrite a decision's substance to match a later change of
mind — record a new decision that supersedes it, and set both
`supersedes` and `superseded_by`. `ckb-decide` owns the format.
`wiki/index.md` alone also carries `kb_schema_version`, currently `"1.7"`
`wiki/index.md` alone also carries `kb_schema_version`, currently `"1.8"`
for the base contract. An installed module may raise it additively by
declaring `requires_kb_schema` and contributing optional types and fields;
`ckb-module` applies that bump at install time. Detailed schema migration

View file

@ -262,7 +262,12 @@ with them), printed into their own session-start brief and nobody else's.
"Remember this" or "lesson: …" records a lesson: a personal one in the
profile, a KB-wide one in `wiki/lessons.md`, which waits for a second
person's review. Precedence is `AGENTS.md`, then a project's `## Rules`, then
the profile. Skill: `.agents/skills/ckb-profile/SKILL.md`.
the profile. Each lesson records the context it was learned in (agent/model,
template version), so after a model or template change the agent rechecks
it instead of treating it as settled. Lessons from KBs under `linked/`
appear in the brief as upstream, ranked below local ones, and a new KB can
start with lessons carried from the one that created it. Skill:
`.agents/skills/ckb-profile/SKILL.md`.
### Kept Answers
An answer that took real work can be kept on request in
@ -287,6 +292,10 @@ what was decided, `decided_by` whom, `decided_on` what date, why, the
alternatives that lost, and what it `affects`. A `status` field
(`proposed`/`accepted`/`rejected`/`superseded`/`reversed`) says whether it is
actually in force, and an optional `review_on` date marks it for revisiting.
An optional `expected_outcome`, written when the decision is made, says what
it should change; at review the agent records what actually happened
(`outcome: met/partly/missed/unclear`) in an appended `## Outcome`, and a
`## Tried` section keeps what was built or tested and dropped on the way.
Decision pages are **append-only**. When the choice changes, a *new* decision
supersedes the old one - `supersedes`/`superseded_by` are set on both sides,
@ -298,7 +307,7 @@ Postgres, and before that MySQL, changed in September because of reporting".
"what decisions touch Y" are direct lookups. The `ckb-decide` skill records
decisions and answers questions about them; `ckb-lint` checks their structure
(status vocabulary, required dates, two-sided supersession, unique numbers,
overdue reviews).
overdue reviews, unrecorded outcomes).
### Query Gaps
If the cascade cannot answer a question, the agent records or proposes a
@ -365,7 +374,7 @@ source or it doesn't. `## Notes` is the inverse guarantee - on pages the agent
regenerates (connector indexes, code maps), it is the one place an annotation
survives the next rebuild.
`wiki/index.md` alone also carries `kb_schema_version` (currently `"1.7"`), declaring
`wiki/index.md` alone also carries `kb_schema_version` (currently `"1.8"`), declaring
which revision of this schema the wiki was authored against - bump minor for
additive optional fields, major for breaking changes.
@ -491,8 +500,12 @@ source citations, and consistency with the rest of the wiki. Pages below
### Error Book
Systematic errors (orphan links, formatting issues, cross-page contradictions)
are recorded in `wiki/error-book.md` with root cause, applied fix, and a
reusable constraint to prevent recurrence. Two-layer repair:
are recorded in `wiki/error-book.md`, one entry per recurring problem. Each
entry names the suspected root cause and keeps every fix tried against it:
what changed, what it was expected to stop (written before the result was
known), what later lint runs actually saw, and the model and template
version it ran under. Fixes that did not hold stay on the record, so the
same fix is not tried again blind. Two-layer repair:
- **Layer 1** - deterministic auto-fix for structural issues
- **Layer 2** - agent reasoning pass for semantic/cross-page issues

View file

@ -274,8 +274,12 @@ Każda osoba może mieć profil w `workload/profiles/` (rola, jak z nią
pracować), wypisywany w podsumowaniu startu jej własnej sesji i niczyjej
innej. „Remember this” albo „lesson: …” zapisuje lekcję: osobistą w profilu,
dla całej KB w `wiki/lessons.md`, gdzie czeka na przegląd drugiej osoby.
Pierwszeństwo: `AGENTS.md`, potem `## Rules` projektu, potem profil. Skill:
`.agents/skills/ckb-profile/SKILL.md`.
Pierwszeństwo: `AGENTS.md`, potem `## Rules` projektu, potem profil. Każda
lekcja zapisuje kontekst, w którym powstała (agent/model, wersja szablonu),
więc po zmianie modelu lub szablonu agent ją sprawdza, zamiast uznawać za
pewnik. Lekcje z baz w `linked/` pojawiają się w podsumowaniu jako
zewnętrzne, poniżej lokalnych, a nowa KB może zacząć od lekcji przeniesionych
z bazy, która ją utworzyła. Skill: `.agents/skills/ckb-profile/SKILL.md`.
### Zachowane odpowiedzi
Odpowiedź, która wymagała prawdziwej pracy, można na prośbę zachować w
@ -303,6 +307,10 @@ ekspert to słabsze twierdzenie niż zapisany.
(`affects`). Pole `status`
(`proposed`/`accepted`/`rejected`/`superseded`/`reversed`) mówi, czy decyzja
faktycznie obowiązuje, a opcjonalna data `review_on` oznacza ją do przeglądu.
Opcjonalne `expected_outcome`, zapisane w chwili podjęcia decyzji, mówi, co
ma ona zmienić; przy przeglądzie agent zapisuje, co faktycznie się stało
(`outcome: met/partly/missed/unclear`) w dopisywanej sekcji `## Outcome`, a
sekcja `## Tried` zachowuje to, co zbudowano lub przetestowano i porzucono.
Strony decyzji są **tylko do dopisywania**. Gdy wybór się zmienia, *nowa*
decyzja zastępuje starą — `supersedes`/`superseded_by` ustawiane są po obu
@ -391,7 +399,7 @@ zgadza, albo nie. `## Notes` daje odwrotną gwarancję — na stronach
regenerowanych przez agenta (indeksy konektorów, mapy kodu) to jedyne miejsce,
w którym adnotacja przetrwa kolejną przebudowę.
Sam `wiki/index.md` dodatkowo zawiera `kb_schema_version` (obecnie `"1.7"`),
Sam `wiki/index.md` dodatkowo zawiera `kb_schema_version` (obecnie `"1.8"`),
deklarujący, według której wersji tego schematu wiki została napisana —
zwiększaj wersję pomniejszą dla dodatkowych opcjonalnych pól, główną dla
zmian łamiących kompatybilność.
@ -542,9 +550,13 @@ są oznaczane do przeglądu lub przepisywane podczas kolejnego lintowania.
### Księga błędów (Error Book)
Systematyczne błędy (uszkodzone linki, problemy z formatowaniem,
sprzeczności między stronami) są zapisywane w `wiki/error-book.md` wraz z
przyczyną, zastosowaną naprawą i ograniczeniem wielokrotnego użytku
zapobiegającym powtórce. Naprawa dwuwarstwowa:
sprzeczności między stronami) są zapisywane w `wiki/error-book.md`, jeden
wpis na powracający problem. Wpis nazywa podejrzewaną przyczynę i zachowuje
każdą wypróbowaną naprawę: co zmieniono, czemu miało to zapobiec (zapisane,
zanim znany był wynik), co faktycznie pokazały kolejne przebiegi lintu oraz
model i wersję szablonu, przy których to działało. Naprawy, które nie
zadziałały, zostają w zapisie, więc ta sama naprawa nie jest próbowana
ponownie na ślepo. Naprawa dwuwarstwowa:
- **Warstwa 1** — deterministyczna automatyczna naprawa problemów
strukturalnych
- **Warstwa 2** — przebieg rozumowania agenta dla problemów

View file

@ -34,14 +34,14 @@ without asking.
## Contents
1. [Current page schema (1.7)](#current-page-schema-kb_schema_version-17)
1. [Current page schema (1.8)](#current-page-schema-kb_schema_version-18)
2. [KB schema version history](#kb-schema-version-history)
3. [Template version history](#template-version-history)
4. [Migrating between versions](#migrating-between-versions)
---
## Current page schema (`kb_schema_version: "1.7"`)
## Current page schema (`kb_schema_version: "1.8"`)
### Frontmatter — every non-reserved page
@ -83,7 +83,10 @@ add:
| `decided_on` | for `accepted`/`rejected`/`reversed` | `YYYY-MM-DD`, the date the call was made. |
| `decided_by` | when known | Comma-separated names. Where genuinely unknown, write `unknown` rather than omitting the field - "we don't know who decided this" is itself worth recording. |
| `affects` | no | Comma-separated wiki paths this decision constrains. |
| `review_on` | no | `YYYY-MM-DD` to revisit. Lint reports these once the date passes. |
| `review_on` | no | `YYYY-MM-DD` to revisit. Lint reports these once the date passes, until an `outcome_checked` on or after it. |
| `expected_outcome` | no | *(1.8)* One line: what the decision should change, written when it is recorded. One added later ends with `(added YYYY-MM-DD)`. |
| `outcome` | at review | *(1.8)* `met` / `partly` / `missed` / `unclear`, against `expected_outcome`. Validated by lint. |
| `outcome_checked` | at review | *(1.8)* `YYYY-MM-DD` of the latest outcome review; the dated entry itself is appended to the body's `## Outcome`. |
Decision records are **append-only**. A decision is never rewritten to match
a later change of mind: record a new one that supersedes it, and both stay on
@ -208,7 +211,7 @@ improving it (in-degree is used as a ranking signal).
| `wiki/index.md` | 1.1 | Routing table; the only page carrying `kb_schema_version` |
| `wiki/overview.md` | 1.1 | High-level map of the KB |
| `wiki/log.md` | 1.1 | Reverse-chronological change log for `wiki/` |
| `wiki/error-book.md` | 1.1 | Systemic issues with root cause and fix |
| `wiki/error-book.md` | 1.1 | Recurring problems (since 1.8: one entry per root cause, with every attempt, its expected and observed result and context) |
| `wiki/entities/` | 1.1 | Typed entity pages |
| `wiki/graph/` | 1.1 | `edges.json` plus the vocabulary in `index.md` |
| `wiki/query-gaps.md` | 1.2 | Questions the wiki could not answer, driving demand-driven ingest |
@ -225,6 +228,21 @@ navigation stays lazy.
## KB schema version history
### 1.8 — 2026-11-18 · improvement memory
[D-0023](decisions/0023-improvement-memory.md), after Google's
Self-Improvement Memory (arXiv 2611.18856). Additive:
- **Error book by problem**: `## E<n> — <root cause>` entries with status
and an attempts table (date, change, Expected, Observed, Context); merged
entries stay as `## E4 → E2` aliases.
- **Decisions**: optional `expected_outcome`, `outcome`, `outcome_checked`;
optional body sections `## Tried` and an append-only `## Outcome`.
- **Lessons**: a `Context:` line (agent/model, template version), optional
`Carried from:` and `Also recorded:` lines.
**Migration** (`ckb-upgrade` Step 7 b5): an old error-book table is kept
under `## Earlier entries`; nothing is backfilled.
### 1.7 — 2026-11-02 · topic summaries
[D-0016](decisions/0016-topic-summaries-and-consolidation.md). A new optional

View file

@ -35,14 +35,14 @@ przechodzi przez wersję główną bez pytania.
## Spis treści
1. [Aktualny schemat strony (1.7)](#aktualny-schemat-strony-kb_schema_version-17)
1. [Aktualny schemat strony (1.8)](#aktualny-schemat-strony-kb_schema_version-18)
2. [Historia wersji schematu KB](#historia-wersji-schematu-kb)
3. [Historia wersji szablonu](#historia-wersji-szablonu)
4. [Migracja między wersjami](#migracja-między-wersjami)
---
## Aktualny schemat strony (`kb_schema_version: "1.7"`)
## Aktualny schemat strony (`kb_schema_version: "1.8"`)
### Frontmatter — każda strona niezastrzeżona
@ -85,7 +85,10 @@ dodają:
| `decided_on` | dla `accepted`/`rejected`/`reversed` | `YYYY-MM-DD`, data podjęcia decyzji. |
| `decided_by` | gdy wiadomo | Nazwiska po przecinku. Gdy naprawdę nie wiadomo, wpisz `unknown` zamiast pomijać pole — „nie wiemy, kto to zdecydował" samo w sobie warto zapisać. |
| `affects` | nie | Ścieżki wiki po przecinku, które ta decyzja ogranicza. |
| `review_on` | nie | `YYYY-MM-DD` do ponownego rozważenia. Lint zgłasza je po przekroczeniu daty. |
| `review_on` | nie | `YYYY-MM-DD` do ponownego rozważenia. Lint zgłasza je po przekroczeniu daty, dopóki nie ma `outcome_checked` w tym dniu lub później. |
| `expected_outcome` | nie | *(1.8)* Jeden wiersz: co decyzja ma zmienić, zapisany przy jej zapisie. Dodany później kończy się `(added RRRR-MM-DD)`. |
| `outcome` | przy przeglądzie | *(1.8)* `met` / `partly` / `missed` / `unclear` względem `expected_outcome`. Sprawdzane przez lint. |
| `outcome_checked` | przy przeglądzie | *(1.8)* `RRRR-MM-DD` ostatniego przeglądu wyniku; sam datowany wpis trafia do sekcji `## Outcome`. |
Rekordy decyzji są **tylko do dopisywania**. Decyzji nigdy nie przepisuje się
pod późniejszą zmianę zdania: zapisz nową, która ją zastępuje, a obie zostają
@ -212,7 +215,7 @@ rankingowym).
| `wiki/index.md` | 1.1 | Tabela routingu; jedyna strona niosąca `kb_schema_version` |
| `wiki/overview.md` | 1.1 | Mapa KB z lotu ptaka |
| `wiki/log.md` | 1.1 | Dziennik zmian `wiki/` w odwrotnej chronologii |
| `wiki/error-book.md` | 1.1 | Problemy systemowe wraz z przyczyną i naprawą |
| `wiki/error-book.md` | 1.1 | Powracające problemy (od 1.8: jeden wpis na przyczynę, z każdą próbą, jej oczekiwanym i zaobserwowanym wynikiem oraz kontekstem) |
| `wiki/entities/` | 1.1 | Typowane strony encji |
| `wiki/graph/` | 1.1 | `edges.json` plus słownik w `index.md` |
| `wiki/query-gaps.md` | 1.2 | Pytania, na które wiki nie umiało odpowiedzieć, napędzające ingest sterowany popytem |
@ -229,6 +232,21 @@ nawigacja pozostaje leniwa.
## Historia wersji schematu KB
### 1.8 — 2026-11-18 · pamięć ulepszeń
[D-0023](decisions/0023-improvement-memory.md), na wzór Self-Improvement
Memory Google'a (arXiv 2611.18856). Addytywnie:
- **Księga błędów według problemów**: wpisy `## E<n> — <przyczyna>` ze
statusem i tabelą prób (data, zmiana, Expected, Observed, Context);
scalone wpisy zostają jako aliasy `## E4 → E2`.
- **Decyzje**: opcjonalne `expected_outcome`, `outcome`, `outcome_checked`;
opcjonalne sekcje `## Tried` i dopisywana `## Outcome`.
- **Lekcje**: wiersz `Context:` (agent/model, wersja szablonu), opcjonalne
`Carried from:` i `Also recorded:`.
**Migracja** (`ckb-upgrade` krok 7 b5): stara tabela księgi błędów zostaje
pod `## Earlier entries`; nic nie jest uzupełniane wstecz.
### 1.7 — 2026-11-02 · podsumowania tematów
[D-0016](decisions/0016-topic-summaries-and-consolidation.md). Nowy opcjonalny

View file

@ -302,6 +302,16 @@ marks the old one superseded - without touching the old page's reasoning.
Anything you leave out, it asks for in one round; if you're mid-flow, say so
and it records what you gave it and tells you which fields it left open.
It also asks what the decision should change and when you'd expect to see
it ("reporting runs in Postgres alone by March"). That goes down *before*
anyone knows the result, as `expected_outcome` with a `review_on` date.
When the date comes, lint reminds you; say "review decision 7" and the agent
records what actually happened (met, partly, missed, unclear) in the
page's `## Outcome`, without touching anything written earlier. Anything
you built or tested and dropped on the way goes under `## Tried`. If a
later decision picks one of those again, the agent points that out and asks
what has changed since.
Then ask about them however you like:
> "What did we decide about the billing database?"
@ -365,7 +375,9 @@ This runs a health check across the whole wiki:
that source, since it tells you who to chase
- pages whose source has actually changed get flagged - see below
- quotes that no longer appear in the source they cite get flagged
- recurring systemic issues get written into `wiki/error-book.md`
- recurring systemic issues get written into `wiki/error-book.md`, one
entry per problem with every fix tried, what each was expected to stop,
and what the following lint runs saw
- credentials committed by accident (keys, tokens, private keys, a password
in a URL) anywhere in `wiki/`, `outputs/`, `workload/` or `raw/`, and any
`<private>` passage that leaked into a wiki page, get flagged - never
@ -951,6 +963,14 @@ Nothing becomes a lesson unless you ask. A lesson that should bind everyone
always belongs in `AGENTS.md`; the agent will suggest that rather than edit
it. Skill: `.agents/skills/ckb-profile/SKILL.md`.
Each lesson notes the context it came from: the agent/model and the template
version. After a model or template change, the agent treats older lessons
as strong hints to recheck rather than settled facts. If your KB links
another KB under `linked/`, that KB's new lessons appear in your brief as
*upstream*; they rank below your own, and "adopt that lesson" copies one in.
When you create a new KB from this one, `ckb-init` offers to carry lessons
and error-book problems across.
### Save a conversation
Every new session gets a title the agent picks from your first message

View file

@ -314,6 +314,15 @@ zastąpioną — nie ruszając jej uzasadnienia. O to, czego nie podasz, dopyta
jednej turze; jeśli jesteś w środku pracy, powiedz to, a zapisze, co ma, i
wskaże, które pola zostawił otwarte.
Zapyta też, co decyzja ma zmienić i kiedy powinno to być widać („raporty
działają na samym Postgresie do marca”). Zapisuje to *zanim* znany jest
wynik, jako `expected_outcome` z datą `review_on`. Gdy data nadejdzie, lint
przypomni; powiedz „review decision 7”, a agent zapisze, co faktycznie się
stało (met, partly, missed, unclear) w sekcji `## Outcome`, nie ruszając
niczego zapisanego wcześniej. To, co po drodze zbudowano lub przetestowano i
porzucono, trafia do `## Tried`. Jeśli późniejsza decyzja znów wybiera coś
stamtąd, agent to zauważy i zapyta, co się od tego czasu zmieniło.
Potem pytaj, jak chcesz:
> „Co zdecydowaliśmy w sprawie bazy danych billingu?"
@ -382,7 +391,9 @@ To uruchamia przegląd kondycji całej wiki:
patrz niżej
- cytaty, których nie ma już w źródle, na które się powołują, zostają
oflagowane
- powtarzające się problemy systemowe trafiają do `wiki/error-book.md`
- powtarzające się problemy systemowe trafiają do `wiki/error-book.md`,
jeden wpis na problem, z każdą wypróbowaną naprawą, tym, czemu miała
zapobiec, i tym, co pokazały kolejne przebiegi lintu
- przypadkowo zacommitowane dane uwierzytelniające (klucze, tokeny, klucze
prywatne, hasło w adresie URL) gdziekolwiek w `wiki/`, `outputs/`,
`workload/` lub `raw/`, a także fragment `<private>`, który trafił na
@ -995,6 +1006,14 @@ Nic nie staje się lekcją, jeśli o to nie poprosisz. Lekcja, która ma zawsze
obowiązywać wszystkich, należy do `AGENTS.md`; agent to zasugeruje, zamiast go
edytować. Skill: `.agents/skills/ckb-profile/SKILL.md`.
Każda lekcja zapisuje kontekst, z którego pochodzi: agenta/model i wersję
szablonu. Po zmianie modelu lub szablonu agent traktuje starsze lekcje jako
mocne wskazówki do sprawdzenia, a nie pewniki. Jeśli twoja KB podpina inną
w `linked/`, nowe lekcje tamtej pojawiają się w twoim podsumowaniu jako
*zewnętrzne* (upstream); są niżej niż twoje, a „adopt that lesson” kopiuje
jedną do ciebie. Przy tworzeniu nowej KB z tej `ckb-init` proponuje
przeniesienie lekcji i problemów z księgi błędów.
### Zapisywanie rozmowy
Każda nowa sesja dostaje tytuł, który agent sam wybiera na podstawie

View file

@ -0,0 +1,125 @@
---
type: decision
tldr: Schema 1.8 keeps the KB's own improvement history the way Google's Self-Improvement Memory does: error-book entries per recurring problem with every attempt, its expected and observed result and its context; decisions with an expected_outcome checked at review and a ## Tried section; lessons with a Context line, upstream lessons from linked KBs, lessons carried into a new KB, and merges that keep aliases. A self-improvement loop over the template is deferred until evals exist; loading the whole record into context unretrieved is rejected.
status: accepted
decided_on: 2026-11-18
decided_by: Michał Kopeć
affects: AGENTS.md, wiki/index.md, wiki/error-book.md, wiki/decisions/index.md, .agents/skills/ckb-lint/, .agents/skills/ckb-decide/, .agents/skills/ckb-profile/, .agents/skills/ckb-init/, .agents/skills/ckb-upgrade/
review_on: 2027-02-18
expected_outcome: By the review, at least one error-book problem in a real KB has two or more attempts with a filled-in Observed column, and at least one decision with an expected_outcome has had its outcome recorded; if neither has happened, the format is too heavy and should be cut back.
confidence: 0.7
quality: 0.8
last_updated: 2026-11-18
freshness_window_days: 365
retention: high
---
# D-0023 — Improvement memory: attempts, expected outcomes, lesson context
**Status:** Accepted · **Decided:** 2026-11-18 · **Deciders:** Michał Kopeć
## Context
Yu et al., *Self-Improvement Memory Enables Continual Recursive
Self-Improvement* (Google, arXiv 2611.18856, 2026-11-13), studies agents
that improve their own harness. When the model or the task changes, the
improved harness alone does not say why each change was made or whether it
worked. Their Self-Improvement Memory (SIM) keeps, per recurring problem,
every attempt with its rationale, the tasks it was **predicted** to fix
(written before evaluation), what it actually fixed, and the setting it ran
in. Continuing with SIM beat continuing with the harness alone in all ten
of their settings (+2.36 points on average across ten paired coding runs,
p = 0.003). The record helped most by keeping the *reason* for an inherited
change, so the next optimizer adapted it instead of reverting it, and by
keeping failed attempts so they were not retried blind.
A Cascade KB already improves itself in small ways: lint records systemic
issues in the error book, people record lessons, decisions record why. At
2.2.0 none of these held SIM's three working parts. The error book was one
row per incident. Nothing wrote down an expected result before the result
was known. Lessons did not say which model or template they were learned
under.
## Decision
Schema 1.8. Every addition is optional and additive.
1. **Error book by problem.** `wiki/error-book.md` holds one entry per
recurring problem (`## E<n> — <root-cause hypothesis>`, status
`open`/`watching`/`resolved`, a `Constraint:` once known), each with a
table of attempts: date, what changed, **Expected** (written first),
**Observed** (`pending`, then `held`/`recurred`/`partly` with a date),
**Context**. Lint check 9 appends to an existing problem before opening a
new one, fills in pending observations on each run, and never rewrites an
attempt. Old single-table error books are kept as `## Earlier entries`.
2. **Expected outcomes on decisions.** `expected_outcome` (one line, written
when the decision is recorded; one added later is marked `(added
YYYY-MM-DD)`), and at review `outcome` (`met`/`partly`/`missed`/`unclear`),
`outcome_checked` and a dated entry appended to `## Outcome`. Lint
validates the vocabulary and treats a review as done once
`outcome_checked` is on or after `review_on`.
3. **Context on lessons and attempts.** A lesson carries `- **Context:**
<agent/model>, template <VERSION>[, setting]`. A lesson from a different
context is a strong hint to recheck, not a rule; a lesson without one is
context unknown.
4. **What was tried.** Decision records gain an optional `## Tried` section,
for what was built or tested and dropped, kept apart from *Alternatives
considered* (weighed, not tried). Template ADRs use it too. Picking a
direction that is listed as tried, or reversing a rejected decision,
makes `ckb-decide` ask what is new since then.
5. **Experience across KBs.** The session brief lists new lessons from each
`linked/<name>/wiki/lessons.md` as `upstream`, ranked below local ones
and read-only. "Adopt that lesson" copies one in with a `Carried from:`
line. `ckb-init`, run inside a populated KB, offers once to carry chosen
lessons and error-book problems into the new KB; carried problems become
`watching`, both files `review: needs_update`. Profiles are never carried.
6. **Merges keep aliases.** A merged error-book entry stays as
`## E4 → E2 — merged YYYY-MM-DD`, and its attempts move across marked
`(from E4)`. A merged lesson is struck through with `merged into "<rule>"`
and the survivor gets `Also recorded:`. Struck-through lessons are no
longer listed in the brief.
## Tried
Nothing was piloted before this decision. The paper's results come from
harness optimisation on benchmarks, not from knowledge bases, so how well
this transfers is the open question. See `review_on` and `expected_outcome`
above.
## Alternatives considered
- **A self-improvement loop for the template** — the paper's own setting:
propose edits to skills, score them on an eval set, check a held-out
split and earlier passing cases, keep the winners, all with SIM recording
the attempts. **Deferred, to consider later.** It needs a behavioural eval
set first (the routing and retrieval evals already listed as gaps). Once
one exists, use the paper's two lessons: hold out part of the eval set,
and check whether previously passing cases still pass after a change.
- **Load the whole record into context, unretrieved** — SIM gives the
optimizer the full memory every round with no retrieval or truncation.
**Rejected.** It works for one optimizer's notes, which stay small. A
KB's error book, lessons and decisions grow without bound and compete
with the context budget lint enforces (check 19). The brief already lists
new lessons; the rest is read index-first when a task bears on it.
- **Machine-readable JSON for the error book**, as SIM stores it.
Rejected: every other KB record is Markdown that people edit and review,
and nothing needs to parse attempts automatically yet.
- **Backfilling expected outcomes and lesson context on upgrade.** Rejected:
a prediction written after the result is hindsight, and a guessed context
is worse than an honest "unknown".
## Consequences
- Recording a decision asks one more question (what it should change), and
lint asks for an outcome when a review comes due. That is more friction;
the review date above checks whether it is used.
- The error book becomes longer and more useful only if lint really goes
back to pending attempts. The skill says so; nothing in a script enforces
it.
- A KB that links others sees their lessons in its brief, which may be
noise if the KBs differ a lot. Upstream lessons rank last and are only
listed, never applied unasked.
## Sources
- `https://arxiv.org/abs/2611.18856` — Yu et al., 2026-11-13, read 2026-11-18 (§3.2, §3.3, App. D.2–D.3, F.6)

View file

@ -10,6 +10,10 @@ but they live here, in the template layer, because:
pages the content layer is allowed to delete.
`ckb-upgrade` carries this folder along with the rest of the template layer.
Since D-0023 these records may carry `expected_outcome`, `## Tried` and an
appended `## Outcome`, like any decision: write what a change should do
before it ships, and check it when `review_on` comes.
`ckb-decide` never writes here — it is not a place for a KB's project decisions.
| # | Decision |
@ -36,3 +40,4 @@ but they live here, in the template layer, because:
| [D-0020](0020-voice-module.md) | Opt-in `voice` module: outgoing prose (kept answers, documents, reports, the project overview) is rewritten in a voice whose profile is a wiki page in `wiki/voice/`; settings in `ckb.yaml`; routed by the `AGENTS.md` block, not per-skill edits; frozen spans never change, vague claims are flagged rather than cut; `writer:` plugs in an external rewriting skill |
| [D-0021](0021-convert-module.md) | Opt-in `convert` module: Markdown becomes Word/ODT/PDF/HTML through pandoc in a named house style, a reference `.docx` in `templates/styles/` built from a few settings or copied from a company `.docx`/`.dotx`; PDF and ODT go through LibreOffice from the styled `.docx`; KB links and frontmatter are cleaned in the copy; the Markdown stays the source |
| [D-0022](0022-dev-branch-and-merge-versioning.md) | All work lands on `dev`; `VERSION` changes only when `dev` is merged into `experimental`, `test` or `main`, a version keeps its number when promoted, `main` is merged back after a release, `1.x` keeps its own fixes, and `dev` is not a channel |
| [D-0023](0023-improvement-memory.md) | Schema 1.8: error book per recurring problem with every attempt, its expected and observed result and context; decisions with `expected_outcome` checked at review and a `## Tried` section; lessons with context, upstream lessons from `linked/`, lessons carried by `ckb-init`, merges with aliases; a self-improvement loop deferred until evals exist; loading the whole record unretrieved rejected |

View file

@ -0,0 +1,109 @@
"""Schema 1.8 (D-0023): decisions carry an expected outcome checked at review,
and the session brief lists upstream lessons from linked KBs and skips
struck-through ones."""
import json
import shutil
import subprocess
import sys
import tempfile
import unittest
from datetime import date
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
FIX = ROOT / "tests/fixtures/minimal-kb"
SCRIPTS = ROOT / ".agents/skills/ckb-lint/scripts"
LINT = SCRIPTS / "lint_report.py"
TODAY = "2026-11-18"
sys.path.insert(0, str(SCRIPTS))
import session_start # noqa: E402
DECISION = ("---\ntype: decision\ntldr: {n}\nstatus: accepted\ndecided_on: 2026-09-01\n"
"decided_by: A\n{extra}---\n\n# D-{n}\n")
class DecisionOutcomes(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.tmp = tempfile.TemporaryDirectory()
kb = Path(cls.tmp.name) / "kb"
shutil.copytree(str(FIX), str(kb))
d = kb / "wiki" / "decisions"
d.mkdir(exist_ok=True)
pages = {
"0091-expects.md": "review_on: 2026-11-01\nexpected_outcome: Fewer orphans.\n",
"0092-checked.md": "review_on: 2026-11-01\nexpected_outcome: Fewer orphans.\n"
"outcome: partly\noutcome_checked: 2026-11-10\n",
"0093-plain-due.md": "review_on: 2026-11-01\n",
"0094-bad-outcome.md": "outcome: great\noutcome_checked: 2026-11-10\n",
"0095-no-outcome.md": "outcome_checked: 2026-11-10\n",
"0096-old-check.md": "review_on: 2026-11-15\nexpected_outcome: X.\n"
"outcome: met\noutcome_checked: 2026-11-10\n",
}
for name, extra in pages.items():
(d / name).write_text(DECISION.format(n=name[:4], extra=extra), encoding="utf-8")
idx = d / "index.md"
old = idx.read_text(encoding="utf-8") if idx.is_file() else "# Decisions\n"
idx.write_text(old + "".join(f"\n- [{n}]({n})" for n in pages) + "\n", encoding="utf-8")
r = subprocess.run([sys.executable, str(LINT), "--json", "--today", TODAY], cwd=str(kb),
capture_output=True, text=True, encoding="utf-8")
cls.findings = json.loads(r.stdout)["trees"][0]["decisions"]
@classmethod
def tearDownClass(cls):
cls.tmp.cleanup()
def about(self, name):
return [f for f in self.findings if name in f]
def test_due_review_with_expectation_asks_for_outcome(self):
(f,) = self.about("0091")
self.assertIn("due for review", f)
self.assertIn("record `outcome`", f)
def test_outcome_checked_after_review_on_closes_it(self):
self.assertEqual(self.about("0092"), [])
def test_plain_due_review_unchanged(self):
(f,) = self.about("0093")
self.assertIn("due for review", f)
self.assertNotIn("expected_outcome", f)
def test_outcome_vocabulary(self):
self.assertTrue(any("`outcome: great`" in f for f in self.about("0094")))
def test_checked_without_outcome(self):
self.assertTrue(any("no `outcome`" in f for f in self.about("0095")))
def test_check_before_review_on_does_not_close_it(self):
self.assertTrue(any("due for review" in f for f in self.about("0096")))
LESSONS = ("---\ntype: lessons\ntldr: Lessons.\n---\n\n# Lessons\n\n"
"### 2026-11-01 — {a}\n- **Scope:** KB\n\n"
"### 2026-11-02 — ~~{b}~~ — dropped 2026-11-05: no longer true\n- **Scope:** KB\n")
class UpstreamLessons(unittest.TestCase):
def test_local_and_upstream_listed_struck_skipped(self):
with tempfile.TemporaryDirectory() as t:
root = Path(t)
(root / "wiki").mkdir()
(root / "wiki/lessons.md").write_text(LESSONS.format(a="Local rule", b="Dead local"), encoding="utf-8")
up = root / "linked" / "parent" / "wiki"
up.mkdir(parents=True)
(up / "lessons.md").write_text(LESSONS.format(a="Parent rule", b="Dead parent"), encoding="utf-8")
out = "\n".join(session_start.lessons_lines(root, None, date(2026, 11, 18)))
self.assertIn("lessons: 1 in wiki/lessons.md", out)
self.assertIn("Local rule", out)
self.assertIn("upstream lessons: 1 in linked/parent/wiki/lessons.md", out)
self.assertIn("Parent rule", out)
self.assertNotIn("Dead", out)
def test_nothing_without_lessons(self):
with tempfile.TemporaryDirectory() as t:
self.assertEqual(session_start.lessons_lines(Path(t), None, date(2026, 11, 18)), [])
if __name__ == "__main__":
unittest.main()

View file

@ -22,4 +22,7 @@ number and its page, and the newer decision points back at it.
## Open questions
Decisions still at `proposed`, and any decision whose `review_on` date has
passed, are the ones worth chasing. `ckb-lint` reports both.
passed without an `outcome_checked` on or after it, are the ones worth
chasing. `ckb-lint` reports both. A decision with an `expected_outcome` is
reviewed by comparing it with what happened (`ckb-decide`, "review
D-NNNN"); the result is appended to its `## Outcome`.

View file

@ -1,15 +1,34 @@
---
type: error-book
tldr: Table of known ingest/lint errors, root causes, and the constraints derived from them.
last_updated: 2026-10-09
tldr: Recurring ingest/lint problems, each with every fix tried against it, what each fix was expected to change, and what it actually did.
last_updated: 2026-11-18
---
# Error Book
Records systematic compilation errors, their root causes,
and the reusable constraints derived from them.
One entry per **recurring problem**, not per incident. Each entry names the
root cause as a hypothesis and keeps every attempt to fix it, in order,
including the ones that did not work. Before opening a new entry, check
whether an existing one already explains the failure and append to it
instead. `ckb-lint` (check 9) owns the format.
| Date | Error | Root Cause | Fix / Constraint |
|------|-------|-----------|------------------|
<!-- Entry format:
*(Agent maintains this during lint and ingest operations.)*
## E1 — <root-cause hypothesis, one line>
- **Opened:** YYYY-MM-DD · **Status:** open | watching | resolved
- **Constraint:** <the rule that should prevent recurrence, once one is known>
| Date | Attempt (what changed, where) | Expected | Observed | Context |
|------|-------------------------------|----------|----------|---------|
| YYYY-MM-DD | <the fix> (`path/changed.md`) | <what it should stop, written before the result is known> | pending | <model/agent>, template <VERSION> |
Observed is `pending` until the next lint run that could show it, then
`held YYYY-MM-DD`, `recurred YYYY-MM-DD: <how>` or `partly YYYY-MM-DD: <how>`.
An attempt is never deleted or reworded after the fact; a later attempt
supersedes it.
A merged entry keeps its number as an alias:
## E4 → E2 — merged YYYY-MM-DD (same root cause)
-->
*(No problems recorded yet.)*

View file

@ -1,5 +1,5 @@
---
kb_schema_version: "1.7"
kb_schema_version: "1.8"
---
# Knowledge Base Index