Cascade Knowledge Base

Read this in: English | Polski

Source repo (always the most up-to-date version): git.wierzbowa.cloud/michal/ckb

A layered, agent-managed wiki where local content overlays read-only upstream sources. Built on Karpathy's LLM Wiki pattern with extensions for scaling, lifecycle management, and multi-agent support.

This document is a technical feature overview. For a task-oriented guide — how to create a wiki, add knowledge, keep it tidy, sync with others, and worked examples for every use case — see MANUAL.md (Polski).

---

Directory Structure

├── libs/         # Read-only external sources — git-copy clones (gitignored) OR
│                 #   connector configs (source.yaml) with a self-contained generated index
├── linked/       # Read-only upstream KBs mounted as symlinks
├── outputs/      # Generated artifacts, exports, compiled files
├── raw/          # User-provided source material
│   └── inbox/    # Drop zone: unprocessed material
├── tmp/          # Temporary files, caches (gitignored)
├── wiki/         # Local structured markdown wiki (agent-managed)
│   ├── index.md      # Routing table with "Use when" triggers + kb_schema_version
│   ├── overview.md   # High-level map
│   ├── log.md        # Root rollup change log
│   ├── error-book.md # Compilation errors & derived constraints
│   ├── query-gaps.md # Failed or missing-answer questions for future ingest
│   ├── projects/     # Optional local query scopes
│   ├── entities/     # Typed entity pages (people, projects, concepts) + own index.md
│   └── graph/        # Edge lists and relationship data + own index.md
└── workload/     # Session summaries & decisions
    └── YYYY-MM-DD_summary.md

---

Cascade Priority

When searching, layers are checked in order — first match wins:

wiki/        (highest)  ← agent writes here, always wins
linked/A/    (medium)   ← symlinked upstream KBs
linked/B/    (low)      ← symlinked upstream KBs
libs/A/      (lowest)   ← git-managed external KB copies, or a connector's own generated index

The agent never writes to linked/ or a git-copy libs/<name>/. To correct upstream content, write the right version in wiki/ — it takes precedence automatically. The one exception is a connector-backed libs/<name>/ (see "External Source Connectors & Indexing" below) — the agent owns and maintains its generated index exactly as it would wiki/.

---

Features

Inbox-Driven Workflow

Drop any raw material (notes, articles, links) into raw/inbox/ without organizing. On "Ingest" (or "Sync the wiki" / "Update the wiki"), the agent processes the inbox — extracts knowledge, files it into wiki/, and archives processed items to raw/ — then reminds you to review the result and run "sync changes" to push it to origin once you're happy with it. Implemented as a Claude Code Skill — see .agents/skills/ckb-ingest/SKILL.md — rather than baked into CLAUDE.md/AGENTS.md, so the full procedure only loads into context when actually invoked. Distinct from the ckb-sync-changes skill, which is a pure git-level operation with no wiki synthesis.

For long conversations, meeting notes, transcripts, or chat exports, ingest uses a structured distillation rather than treating the whole file as one undifferentiated blob: searchable question, short summary, resolution or decision, systems/code references, people involved, and high-signal excerpts that deserve to stay findable on their own.

"High-signal" is an explicit test, not a judgment call, or every excerpt looks worth keeping and the page becomes a second copy of the transcript. An excerpt earns its own retrievable section only if it carries a term rare across the wiki (checked with rg -c — a distinguishing search handle, not a word already on twenty pages), runs to roughly 200 characters or more, and is corroborated by something later in the material agreeing with, acting on, or correcting it. Fail any one and the content still lands in the page, just inside the summary rather than as its own unit. Promoted excerpts carry their parent heading or thread question with them, so they read unambiguously alone.

Lazy-Loading Index with "Use When" Triggers

wiki/index.md is a routing table. Each entry has a Use when column listing trigger keywords. The agent reads the index first (stays in context), matches keywords against the task, and only loads matching pages. This drops context overhead from ~12K to ~3.2K tokens per task.

TLDR-First Query Layer

Every page carries a one-sentence tldr in frontmatter. When querying, the agent reads TLDRs first. If the TLDR already answers the question, the full body is never loaded. Fallback chain: TLDR → body → raw source.

Local Project Scopes

For recurring teams, clients, systems, or initiatives, the wiki can keep plain Markdown scope pages under wiki/projects/. A scope page lists the wiki pages, entities, raw/archive sources, connector-backed libs, outputs, and graph areas that should be searched first for that project. This gives the same practical benefit as a project workspace in a larger retrieval system while staying local, transparent, and editable with any text editor.

Scopes only narrow the first pass. If the scoped search does not answer the question, the agent falls back to the full cascade.

Local Hybrid Retrieval

When index/TLDR routing is not enough, the agent can combine several local signals before answering:

• exact text search with rg for error strings, commands, flags, filenames, hostnames, ticket IDs, and other literal tokens — including across raw/inbox/, so material dropped an hour ago and not yet ingested can still answer a question (and flags that an ingest is overdue)
• semantic/entity matches from page titles, TLDRs, project scopes, and graph relationships
• freshness and confidence metadata, so stale or weak pages are treated with care
• context expansion around a matched section, so answers are grounded in the neighboring headings and paragraphs rather than a lone snippet

Each signal produces its own ranked list, and the lists are then fused rather than resolved by picking a favourite: every candidate scores weight / (k + rank) summed across the lists it appears in, so a page ranked third by three signals beats one ranked first by a single signal. k is 10, deliberately smaller than the 60 rank fusion is usually quoted with — 60 is tuned for retrievers returning hundreds of candidates and flattens all scores into near-identical values against the dozen a local wiki produces. Literal-token queries up-weight the exact-match list, since no amount of title similarity should outrank a match on the exact string someone pasted.

Fused candidates are then deduplicated by claim — a wiki page, the raw/archive/ file it cites, and a connector page pointing back at it are three hits for one fact, not three sources — and reranked 0–10 on how well each answers the literal question rather than how well it matches the question's wording. Same agent, deliberate second pass, no separate model.

The result is normalized internally as an evidence packet: source path, matched claim, date/freshness, confidence/quality, relationship or scope hints, and which signals each candidate was fused from. No server, vector database, or dedicated client is required.

Answer Caveats

Metadata the wiki already tracks is stated in the answer itself, not just consulted while building it. When a page grounding an answer is past its freshness_window_days, carries low confidence/quality, rests on un-ingested raw/inbox/ material, or was checked against a cached connector index rather than a live re-fetch, the answer says so next to the claim it qualifies. Conflicts between two live pages are surfaced the same way even when neither carries superseded_by yet. The failure mode this closes is answering confidently from a stale page without passing that on.

Expertise & Ownership Lookups

"Who knows about X" and "who owns X" are direct graph lookups rather than full-text guesses. Ingest records has_expertise_in edges when someone demonstrably answers questions or explains decisions on a topic, and owns edges for declared responsibility over a system, area, or decision — both from demonstrated evidence only, never inferred from meeting attendance or job title. Where no edge exists yet, retrieval falls back to authorship evidence and says which of the two grounded the answer, since an inferred expert is a weaker claim than a recorded one.

Query Gaps

If the cascade cannot answer a question, the agent records or proposes a short entry in wiki/query-gaps.md: what was asked, where it looked, and the smallest source or page that would close the gap. This makes failed searches useful demand signals for the next ingest instead of disappearing into chat history.

Page Frontmatter Schema

Every wiki page uses YAML frontmatter. type is required; the rest are optional:

---
type: concept              # REQUIRED. Open string: person, project, concept, library, decision, playbook, ...
resource: https://...      # Canonical URI to the authoritative external source this page describes
tldr: One-sentence summary optimised for LLM reading
confidence: 0.0–1.0      # Source corroboration score
quality: 0.0–1.0          # Self-evaluation (below 0.7 → flagged)
supersedes: path/to/old.md
superseded_by: path/to/new.md
last_updated: YYYY-MM-DD
freshness_window_days: 90 # Days before considered stale
retention: high|medium|low
---

• type — required; unregistered string, new values always valid, readers tolerate unrecognized ones
• resource — optional pointer to the live/authoritative source, kept separate from the wiki's own commentary
• confidence — set on write, decays with time, reinforced by new sources
• quality — self-scored on write, pages below 0.7 flagged for review
• supersedes / superseded_by — when new info replaces old, link them
• freshness_window_days — pages older than this get flagged during lint
• retention — low pages are archived after 2× freshness window

wiki/index.md alone also carries kb_schema_version (e.g. "1.3"), declaring which revision of this schema the wiki was authored against — bump minor for additive optional fields, major for breaking changes.

Entity Extraction & Knowledge Graph

During ingest, the agent extracts typed entities (people, projects, libraries, concepts, systems) and stores them as pages in wiki/entities/. Typed relationships are recorded in wiki/graph/edges.json — structural ones (uses, depends_on, caused, contradicts, supersedes) plus people-to-topic ones (has_expertise_in, owns). Queries can walk the graph to discover connected pages (e.g. "what depends on Redis?") or to answer "who knows about X" directly.

Recursive Index & Log Convention

Any wiki/ subdirectory that groups multiple pages (entities/, graph/, future topic folders) keeps its own index.md — a plain link list, no frontmatter — so subdirectory navigation stays lazy instead of requiring a full scan. A subdirectory can also keep its own log.md once it has enough independent history; wiki/log.md stays the root-level rollup and never duplicates a change a subdirectory log already recorded.

External Source Connectors & Indexing (on demand)

A libs/<name>/ folder supports a second population mode alongside the existing git-copy one: a small user-authored libs/<name>/source.yaml declaring a live external source — a SharePoint folder, a Google Drive folder, a plain URL, or another connector — that you don't want to fully mirror locally:

connector: sharepoint
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Finance team's shared reports folder"
refresh_interval_days: 7   # optional, defaults to 30

refresh_interval_days tunes cadence per source — a folder that churns daily deserves a shorter window than a quarterly archive that barely moves — and sets the freshness_window_days stamped on that source's generated pages. Both "index external sources" and "Lint" report a source that's overdue against it, and say by how much, so a read-only user knows which admin to chase rather than quietly trusting a copy three weeks past due.

Say "index external sources" and the agent walks it, resolving connector to whatever live tool is available that session (a connected Microsoft 365/Google Drive MCP tool, or WebFetch for a plain URL), and builds a self-contained generated index inside that same libs/<name>/ — index.md/entities//graph//log.md, mirroring wiki/'s own shape via the Recursive Index & Log Convention above, but scoped entirely to that one connector. This is a deliberate design choice: the index is not blended into the main wiki/entities//wiki/graph/edges.json — it stays separated at the libs/ cascade layer, the same way a git-cloned KB's own files already are. source.yaml itself stays user-only, never written by the agent.

Two refinements on top of that:

• Shared, pre-built indexes. source.yaml can add an optional index: block declaring where the already-built index lives — a git repo, or a shared resource such as a network path or another connector-reachable location:

  index:
    store: git   # git | shared
    location: "https://github.com/org/finance-index-cache.git"
    ref: main    # optional — branch, tag, or subpath hint within that store
  

  Every run checks that location: if it already has an index, fetch it — most people just read what's already there instead of building it themselves. If it doesn't yet, that's the normal first-time state, not an error: a write-access user's very next run is what creates and publishes it there, with no separate "initialize" step.
• Read vs. write, per user, per source. Whether this user can actually rebuild an index (versus only read a fetched/published one) is a separate, local, gitignored libs/<name>/source.local.yaml — read-only by default. Setting access: write there opts a given machine/user in as that source's admin, so a team can designate one or two people to maintain a source while everyone else just reads the result — no redundant rebuilding, no need for every user to have their own connector authorization.

Implemented as a Claude Code Skill — see .agents/skills/ckb-index-external/SKILL.md.

Dual-Linking (Wikilinks + Markdown)

Every cross-reference uses both [[Wikilinks]] (Obsidian-compatible) and standard [markdown](path.md) links. Works in Obsidian graph view, GitHub rendering, and CLI tools. Upstream references use full relative paths: linked/<name>/... or libs/<name>/.... Intra-wiki references prefer project-root-absolute paths (/wiki/entities/foo.md) over relative ones, so links survive later file moves.

Self-Healing Lint

Periodically (or on demand), the agent health-checks the wiki:

• Conformance — flags any page missing parseable frontmatter or a type field
• Freshness — flags pages past their freshness_window_days
• Confidence decay — reduces confidence on unreinforced pages
• Retention sweep — archives retention: low pages past 2× window
• Supersession detection — finds contradictions, links old→new
• Orphan detection — finds pages with no inbound links
• Graph consistency — verifies all edges point to existing entities
• Index/log consistency — verifies every subdirectory has an index.md and no change is double-logged
• Connector cadence — flags a connector-backed source whose generated index is overdue against its refresh_interval_days, and by how much
• Error Book — records systemic issues with root cause and fix

Auto-fixes what it can (broken links, missing backlinks, stale flags), and reminds you to review the result and run "sync changes" to push it to origin once you're happy with it. Implemented as a Claude Code Skill — see .agents/skills/ckb-lint/SKILL.md — rather than baked into CLAUDE.md/AGENTS.md, so the full checklist only loads into context when actually invoked.

Conflict Resolution (Supersession)

When new information contradicts an existing page, the agent adds supersedes / superseded_by links. The old page is preserved but marked stale. Version control for knowledge, not just files.

Quality Scoring

Every page gets a quality score (0.0–1.0) on write, based on structure, source citations, and consistency with the rest of the wiki. Pages below 0.7 are flagged for review or rewritten in the next lint pass.

Error Book

Systematic errors (orphan links, formatting issues, cross-page contradictions) are recorded in wiki/error-book.md with root cause, applied fix, and a reusable constraint to prevent recurrence. Two-layer repair:

• Layer 1 — deterministic auto-fix for structural issues
• Layer 2 — agent reasoning pass for semantic/cross-page issues

Automation Hooks

• New source → auto-ingest on next "Ingest" command
• Session start → load index + latest workload summary; check for unsynchronized changes (uncommitted work, or ahead/behind origin) and suggest ckb-sync-changes if any are found
• Session end → compress observations into workload/; re-check for unsynchronized changes (including anything the session itself just created) and suggest ckb-sync-changes if needed
• Query → file back valuable answers as wiki pages
• Memory write → check contradictions, trigger supersession
• Schedule → periodic lint, consolidation, retention decay

Demand-Driven Context (DDC)

The wiki grows based on actual agent failures rather than upfront curation:

1. Agent can't answer → identifies missing knowledge
2. Proposes minimal entity/page to fill the gap
3. User approves or provides source material
4. Next ingest incorporates it

Converges to a stable KB after ~20–30 cycles.

Session Summaries

After every conversational action, the agent appends to workload/YYYY-MM-DD_summary.md. This provides continuity between sessions and a browsable history of how the KB evolved. The agent reads the latest summary on session start to pick up where it left off.

Change Log

Every wiki modification is immediately logged in wiki/log.md in reverse chronological order (most recent first), recording what changed, why, and the source.

OKF Export (on demand)

The wiki can be exported as an Open Knowledge Format v0.1-conformant bundle at outputs/okf/, consumable by any generic OKF tool (e.g. Google's reference graph visualizer) without disturbing the richer internal schema (confidence/quality/retention/supersedes/dual-linking) that OKF doesn't natively understand. Implemented as a Claude Code Skill — see .agents/skills/ckb-export-okf/SKILL.md — rather than baked into CLAUDE.md/AGENTS.md, so the mapping ruleset only loads into context when actually invoked. outputs/okf/ is gitignored — it's a fully-regenerated build artifact, so each machine/tool regenerates it on demand rather than carrying it in git history.

Starlight Export (on demand)

The wiki can also be exported into an Astro + Starlight-consumable form at outputs/starlight/, producing a human-readable documentation website — unlike the OKF export, which targets machine/tool consumption. A deterministic Python script (scripts/export_starlight.py) handles frontmatter remapping, dual-link collapsing, wikilink resolution, asset copying, and sidebar generation; the agent's job is just to ask the two setup questions (export scope: full runnable scaffold vs. content-only; whether to include the log/error-book meta pages) and relay the script's report. Also on-demand and skill-only — see .agents/skills/ckb-export-starlight/SKILL.md. Like outputs/okf/, outputs/starlight/ is gitignored as a regenerated build artifact.

Guided Onboarding Tours (on demand)

Ask "onboard me on X" (or "where do I start with X", "mini tour of X") to get a short, read-only guided reading order: an overview paragraph plus an ordered list of wiki pages to read, built by walking the knowledge graph outward from the best-matching page (foundations first, then the topic itself, then what builds on it). Never writes to wiki/. See .agents/skills/ckb-onboard-me/SKILL.md.

Project Summary (on demand)

Ask for "a project summary" (or "where do things stand", "catch me up on the project") to regenerate PROJECT-OVERVIEW.md at the repo root — a one-to-two-page snapshot (overview, project state, actions & status, risks, assumptions) synthesized entirely from the current wiki/ contents and its knowledge graph. Always overwritten in full on re-run, never hand-appended to. See .agents/skills/ckb-project-summary/SKILL.md.

Git Sync (on demand)

This repo's own git history can be reconciled with its origin remote on demand: local changes get committed, remote changes get pulled and merged, any conflicts are presented to the user file-by-file to resolve, then the result is pushed automatically. Say "sync changes" to trigger it. Also implemented as a Claude Code Skill — see .claude/skills/ckb-sync-changes/SKILL.md — and deliberately distinct from the content-level "Sync the wiki" / "Ingest" workflow, which processes raw/inbox/ into structured wiki/ pages and has nothing to do with git.

Quiz Mode (on demand)

Ask to be quizzed on the wiki ("quiz me on X", "test my knowledge") for a one-off, scored knowledge check: the agent reads the relevant pages, generates open or multiple-choice questions grounded in specific wiki facts, runs them one at a time with immediate feedback and a running score, and closes with a verdict. Stateless — nothing is saved between runs. See .agents/skills/cbk-quiz/SKILL.md.

Guided Teaching Curriculum (on demand)

Ask to be taught the wiki ("teach me the wiki", "teach me about X", "run a teaching session") for a stateful course rather than a one-off quiz. First call plans it: scopes the material (optionally supplementing thin spots from the web, clearly marked as non-authoritative), asks whether it should be one session or a spaced series (duration, frequency, optional calendar .ics reminders), chunks the content into session-sized portions — preferring an extra session over cramming — and saves the accepted plan and a progress tracker to outputs/teaching/<topic>/. Later calls compare plan against progress, teach the next portion using a different technique each time (Socratic questioning, analogies, worked examples, teach-back, mnemonics, ...), spot-check retention, and re-teach weak spots before advancing. Never writes to wiki/. See .agents/skills/ckb-teach-me/SKILL.md.

---

Quick Start

1. Mount upstream KBs:

   ln -s /path/to/other-kb ./linked/my-upstream
   git clone https://github.com/org/external-kb ./libs/external-kb
   

   Or, for a live source you don't want to fully mirror, drop a libs/<name>/source.yaml instead (see "External Source Connectors & Indexing" above) and say "index external sources."

2. Drop raw material into raw/inbox/ (notes, links, articles).

3. Tell an agent to "Ingest" — it processes the inbox, consults the cascade, extracts entities, and writes structured markdown into wiki/.

4. Ask questions — the agent uses the index for routing, TLDRs for quick answers, and the graph for relationship discovery.

5. Periodically ask to "Lint" — the agent health-checks everything, auto-fixes what it can, and reports issues.

---

Agent Instruction Files

File	Purpose
AGENTS.md	Full instruction for any AI coding agent
CLAUDE.md	Symlink to AGENTS.md, auto-detected by Claude Code

Skills live in one shared location, .agents/skills/, so any agent tooling that reads that convention picks them up. .claude/skills is a symlink to .agents/skills — Claude Code sees the same skill set without a second copy to keep in sync.

---

Tips

• Upstream KBs (linked/ and git-copy libs/) are never modified by agents. A connector-backed libs/<name>/ (one with a source.yaml) is the one exception — the agent owns and maintains its generated index, but only for a user who's opted themselves into access: write locally (see the next point); everyone else's copy stays read-only.
• To correct upstream content, write the correct version in wiki/ — it wins.
• Use raw/inbox/ for anything unprocessed; the agent clears it on ingest.
• The wiki/index.md routing table is the most important file — keep it current.
• Confidence, quality, and freshness let you trust the right content and flag the rest for review.
• The tmp/ directory is gitignored, and so is most of libs/ — but not all of it: a git-copy libs/<name>/'s cloned content stays gitignored as before, while a connector-backed libs/<name>/'s source.yaml and its generated index.md/entities//graph//log.md are tracked, since they're synthesized knowledge worth sharing via "sync changes," not a disposable build artifact. libs/<name>/source.local.yaml (per-user read/write setting) is the one exception that stays gitignored right alongside them — it's personal machine state, never meant to sync. outputs/ itself is tracked, but its regenerated build subdirectories, outputs/okf/ and outputs/starlight/, are gitignored — each is fully reproducible from wiki/ on demand, so there's nothing to reconcile by carrying it in git history. outputs/teaching/ (per-user learning plans and session progress from the teaching skill) is also gitignored, since it's personal session state rather than shared KB content. Commit other, hand-maintained artifacts under outputs/ as normal.

---

Version & License

Current template version: VERSION. Licensed under the Apache License 2.0.