Compare commits

...

4 commits

Author SHA1 Message Date
Michał Kopeć
f3dbce7327 Backport the check-then-fetch-or-create nuance into README/MANUAL docs
The last skill clarification (check the shared index store on every
run; fetch if it exists; a write-access user's run creates it there
the first time it's empty) hadn't been reflected in the user-facing
docs, which only described the steady-state case. Also documents the
optional index.ref field, mentioned in the skill schema but missing
from all four docs' examples.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 08:41:14 +02:00
Michał Kopeć
dc9f990bdd Clarify create/download/update/publish decision table for shared indexes
Spell out explicitly, per run: check whether the shared index store
already has a published index; if so, fetch it regardless of access
level; a write-access user always continues to rebuild and publish
back to that same store, and this is precisely what creates it there
the first time the store is empty. A read-only user never rebuilds or
publishes anything, whether or not the store already had content.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 16:22:55 +02:00
Michał Kopeć
946619de89 Add shared pre-built indexes and per-user read/write access for connector sources
source.yaml gains an optional index: block declaring where an
already-built index lives (a git repo or a shared resource), so a
user can fetch it instead of scanning the live connector from
scratch. Whether a given user may actually rebuild/publish an index
is now a local, per-user, gitignored source.local.yaml (access:
write|read) that defaults to read-only, letting a team designate one
or two admins per external source instead of everyone redundantly
re-indexing it. ckb-lint's checks against a connector's generated
index now respect the same read/write gate.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 16:20:06 +02:00
Michał Kopeć
55c8c352d1 Add connector-backed libs/ with self-contained external source indexing
libs/<name>/ now supports a second population mode alongside git-copy
clones: a user-authored source.yaml declares a live external source
(SharePoint, Google Drive, a plain URL, or another connector), and the
new ckb-index-external skill builds a self-contained generated index
for it (index.md/entities/graph/log.md), scoped entirely to that
connector and never blended into the main wiki/.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 15:56:37 +02:00
11 changed files with 823 additions and 45 deletions

View file

@ -135,7 +135,7 @@ the report rather than silently patching the export.
Before reporting done, re-check the *generated* `outputs/okf/` bundle
against OKF's own conformance criteria (the same shape of check `CLAUDE.md`
§5 step 1 runs against the source wiki):
§6 step 1 runs against the source wiki):
- Every non-reserved `.md` file has frontmatter with a non-empty `type`.
- The root `index.md`'s frontmatter contains only `okf_version` (or is
@ -168,7 +168,7 @@ Tell the user:
spec-valid, don't skip the directory entirely.
- **A wiki page missing `type`:** per Step 3, use `type: unknown` and flag
it in the report — this indicates the source wiki itself failed lint's
conformance check (see `CLAUDE.md` §5 step 1), which is worth surfacing
conformance check (see `CLAUDE.md` §6 step 1), which is worth surfacing
to the user rather than quietly masking it in the export.
- **A future `wiki/archived/` directory:** export it like any other
subdirectory (mirror the structure, apply the same per-file rules) —

View file

@ -0,0 +1,314 @@
---
name: ckb-index-external
description: Walk every connector-backed libs/<name>/ (identified by a source.yaml declaring a live external source — SharePoint, Google Drive, a plain URL, or another connector) and build/refresh a self-contained generated index for it — index.md/entities/graph/log.md, mirroring wiki/'s own shape but scoped entirely to that one connector. The index can optionally be published to (and fetched from) a shared location — a git repo or a shared resource — declared in source.yaml, so most users just read an already-built index instead of re-scanning the live source. Read vs. write access to a given source is a per-user, local-only setting (libs/<name>/source.local.yaml, gitignored) — read-only by default, so one or two designated admins can maintain a source for everyone else. Never touches wiki/, never touches source.yaml, never touches a git-copy lib. Use when the user says "index external sources", "index libs", "refresh the external index", or "scan the SharePoint/Drive folder". This is the on-demand workflow from CLAUDE.md/AGENTS.md §4, distinct from `ckb-ingest` (which processes raw/inbox/ into wiki/) and from a plain git-copy libs/<name>/ (a frozen clone, never touched by any skill).
---
# Index external sources skill
## Purpose
Give `libs/<name>/` a second population mode alongside the existing
git-copy one: a `libs/<name>/source.yaml` declares a *live* external
source — a SharePoint folder, a Google Drive folder, a plain URL, or
another connector — that the user doesn't want to fully mirror locally.
This skill walks that source and builds a small, self-contained index of
what's there: one entity page per document found, plus the same
`index.md`/`graph`/`log.md` shape `wiki/` already uses, rooted at
`libs/<name>/` instead.
This is deliberately **not** blended into the main `wiki/entities/` or
`wiki/graph/edges.json` — the generated index lives entirely inside its own
`libs/<name>/`, at the lowest cascade layer, the same way a git-cloned
KB's own files would. If something in it needs to override or correct what
the connector says, that's what writing the corrected version into `wiki/`
is for (cascade priority already covers that — `wiki/` always wins).
Two refinements on top of that base design:
- **Shared, pre-built indexes.** `source.yaml` can declare *where the
already-built index lives* (a git repo, or a shared resource such as a
network path or another connector-reachable location). Every run checks
that location: if it already has an index, fetch it; if it doesn't yet,
a write-access user's run is what creates it there for the first time.
Either way, a read-only user just gets whatever's already there — they
never have to build the index from scratch themselves.
- **Read vs. write access, per user, per source.** Building/refreshing an
index from the live connector is gated by a **local, per-user** setting
(`libs/<name>/source.local.yaml`, never committed) — read-only by
default. This lets a team designate one or two people as the "admin" of
a given external source, while everyone else just consumes whatever
index the admin has already published, without needing their own
connector authorization or triggering redundant/conflicting rebuilds.
This skill only ever writes within a connector-backed `libs/<name>/` (never
`source.yaml` itself, never a git-copy lib, never anything under `wiki/`)
— and it never touches this repo's own git remote; pushing the resulting
changes is a separate, explicit step (see Step 7).
## Trigger phrases
Use this skill when the user says:
- "Index external sources"
- "Index libs"
- "Refresh the external index"
- "Scan the SharePoint/Drive folder" (or similarly names a specific
connector)
- "Make me the admin for `<source>`" / "give me write access to `<source>`"
/ "I don't want write access to `<source>` anymore" — a lighter-weight
request that only touches that one source's `source.local.yaml` (see
Configuration below) and doesn't necessarily trigger a full re-index in
the same turn unless the user also asks for one.
Do **not** use this skill for "Ingest" / "Sync the wiki" (that's
`ckb-ingest` — raw material into `wiki/`, unrelated to `libs/`) or for
"Lint" (that's `ckb-lint`, which now also health-checks a connector's own
generated index, but doesn't build it in the first place).
## Configuration
### `libs/<name>/source.yaml` (shared, tracked in git, same for everyone)
```yaml
connector: sharepoint # sharepoint | google_drive | web | custom — open string, not a fixed enum
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Finance team's shared reports folder" # optional
index: # optional — omit entirely for the original default: the index lives only
# here, in this repo, tracked by this repo's own git (nothing to fetch/publish)
store: git # git | shared — open string, where the built index is published to / fetched from
location: "https://github.com/org/finance-index-cache.git" # a git repo URL (store: git), or a
# filesystem/network path or connector-reachable location (store: shared)
ref: main # optional — branch, tag, or subpath hint within that store
```
### `libs/<name>/source.local.yaml` (per-user, per-machine, **never committed**)
```yaml
access: write # write | read — defaults to read if this file is absent or the field is omitted
```
This file is intentionally **not** carved out of `.gitignore` — it stays
caught by the existing `libs/*/*` catch-all, exactly like any other
unlisted file inside a connector-backed `libs/<name>/`. It's local
machine/user state, the same category as a personal `.env` — never synced,
never reviewed by anyone else, and it's what makes "read vs. write" a
per-user decision rather than a shared one baked into `source.yaml`.
Unlike `source.yaml`, this skill *may* create or edit `source.local.yaml`
on this user's behalf — but only when they explicitly ask to become (or
stop being) a given source's admin (e.g. "make me the admin for the
finance reports source" / "I don't want write access to X anymore"), never
as a side effect of just running "index external sources."
## How to run this skill
### Step 1 — Find connector-backed libs
Scan `libs/*/` for a `source.yaml`. A `libs/<name>/` with one is
connector-backed and in scope; a `libs/<name>/` without one is a git-copy
lib and stays completely untouched by this skill.
If no `libs/<name>/source.yaml` exists anywhere, tell the user there's
nothing to index yet and how to add one (a `source.yaml` with `connector`
and `location`), and stop — don't fabricate an index from nothing.
### Step 2 — Read the local access setting
Read `libs/<name>/source.local.yaml` if it exists. `access: write` means
this user is an admin for this one source; anything else (the field is
`read`, or the file doesn't exist at all) means read-only. This check is
**per connector-backed lib** — a user can be write-access for one source
and read-only for another.
### Step 3 — Check the shared index store, and fetch it if it exists
If `source.yaml` has an `index:` block, check whether `index.location`
already has a published index (per `index.store`: a `git` remote with
commits/a reachable ref, or a filesystem/network/`shared` location with
files already in it):
- **It exists** → fetch/pull it (`git pull`/clone for `git`, a plain file
copy for a filesystem/network `shared` path, or the matching connector
tool for a `shared` location that needs one) and use it to refresh
`libs/<name>/{index.md,entities/,graph/,log.md}`. Do this **regardless
of this user's access level** — reading the published index never
requires write access, and it's exactly what lets a read-only user skip
building anything themselves.
- **It's empty / nothing published yet** → there's nothing to fetch. Don't
treat this as an error; it's the normal first-time state before anyone
with write access has run this. Continue to Step 4 — whether that turns
into "create it" or "nothing to report" depends entirely on this user's
access level.
If no `index:` block is configured at all, skip this check — the index (if
any exists yet) already lives directly in `libs/<name>/`, same as before.
### Step 4 — Decide whether to build or refresh from the live connector
This is the one place access level actually changes behavior:
- **Read-only** (the common case, and the default for anyone who hasn't
set `access: write` locally): stop here, regardless of what Step 3
found. Never touch the live connector on a read-only user's behalf.
- If Step 3 fetched something (or one already lived in `libs/<name>/`
with no `index:` block): report what it covers.
- If nothing exists anywhere yet (Step 3 found the shared store empty,
or there's no `index:` block and `libs/<name>/` is empty too): say so
plainly, and suggest asking whoever administers this source to run it,
or setting `access: write` locally if this user is meant to be that
admin.
- **Write access**: always continue to Step 5, whether Step 3 fetched an
existing index (refresh it) or found the store empty (build the very
first version from scratch) — the two cases are handled identically
from here on; Step 7 is what actually creates the remote copy either
way.
### Step 5 — Resolve the connector and enumerate/summarize documents
Resolve `connector` to whatever live tool is actually available this
session:
- `sharepoint` / a Microsoft-hosted location → search (`ToolSearch`) for a
connected Microsoft 365 MCP tool.
- `google_drive` → search for a connected Google Drive MCP tool.
- `web` / `custom` / anything else with a plain URL → `WebFetch`.
If the needed connector isn't authorized in this session (per the current
MCP auth state), **don't fail the whole run** — tell the user which
connector needs authorizing (via claude.ai connector settings, or `claude
mcp`/`/mcp` for non-claude.ai servers), skip that source, and continue with
any others.
For each document found at the location:
- Read enough content to write a real 1-2 sentence `tldr` when the
connector allows fetching content.
- If only metadata is available (listing only, no read access), say so
honestly in the `tldr` (e.g. "Metadata only — read access not granted")
and set `confidence` low (e.g. 0.3) rather than fabricating a summary.
Create/update one entity page per document at
`libs/<name>/entities/<doc-slug>.md`:
```yaml
---
type: document
resource: <document's own URL/path>
tldr: ...
confidence: 0.30.9 # per the rule above
quality: ...
last_updated: YYYY-MM-DD
freshness_window_days: 30 # shorter than a typical wiki page — external sources change without notice
retention: medium
---
```
### Step 6 — Update the hub page, graph, index, and log
Create/update `libs/<name>/index.md` — the root routing page for this
connector, mirroring `wiki/index.md`'s own role:
```yaml
---
type: external-source
resource: <location, from source.yaml>
tldr: One sentence — what this source is and how many documents were found
last_updated: YYYY-MM-DD
---
```
followed by a short routing table pointing to `entities/index.md` and
`graph/index.md`.
Record any real relationships between documents (e.g. folder hierarchy) as
edges in `libs/<name>/graph/edges.json`, using a `contains`/`part_of`-style
edge type — this is fine to be thin or empty for a flat source with no
useful structure beyond a document list.
Update `libs/<name>/entities/index.md` and `libs/<name>/graph/index.md`
(flat bullet lists, no frontmatter, same convention as `wiki/entities/
index.md`/`wiki/graph/index.md`). Log every created/updated page in
`libs/<name>/log.md`, same reverse-chronological format as Rule B
(`CLAUDE.md`/`AGENTS.md` §7) — this log is independent of `wiki/log.md`;
**nothing under `wiki/` is touched by this skill at all.**
### Step 7 — Publish, if this source has both write access and a configured index store
If this user has `access: write` **and** `source.yaml` has an `index:`
block, push the refreshed `libs/<name>/{index.md,entities/,graph/,log.md}`
back out to `index.location` (`git push` for `store: git`, a file copy
back for a filesystem/network `store: shared` path, or the matching
connector tool for a `shared` location that needs one) — **every single
run**, not just the first one, so every other user's next fetch (Step 3)
picks up the update. This is exactly how a shared index gets created in
the first place: if Step 3 found the store empty, this step's push is what
populates it for the first time; there's no separate "initialize" action.
If no `index:` block is configured, there's nothing external to publish;
the refreshed files staying inside this repo's own `libs/<name>/` (tracked
by this repo's own git) *is* the publish step, same as the original
default behavior.
If this user is **read-only**, this step never runs — nothing is ever
pushed or copied to `index.location` (or, in the no-`index:`-block case,
nothing is even rebuilt in `libs/<name>/` to begin with). A read-only user
has, by construction, nothing of their own to save back — Step 4 already
stopped them before anything was built.
### Step 8 — Remind to review and sync
This is always the last step, every time this skill makes any change at
all. Close with a short reminder — do not sync or push anything yourself
here:
> "External source indexing complete for `libs/<name>/` — see
> `libs/<name>/log.md` for the details. Please review the changes, and
> once you're happy with them, say 'sync changes' to commit and push them
> to `origin`."
If nothing was actually indexed or fetched (no connector-backed libs
found, every source was skipped for lack of authorization, or every source
was read-only with nothing new to fetch), skip this reminder.
## Edge cases
- **No connector-backed `libs/<name>/` exists** — report that and explain
how to add one; don't touch anything.
- **`source.local.yaml` doesn't exist yet for a source** — treat as
read-only; this is the correct default, not a misconfiguration worth
flagging loudly. Mention once, briefly, how to become the admin
(`access: write` in that file) if the user seems to expect a rebuild.
- **Everyone is read-only and nothing has ever been published** — report
that clearly rather than silently doing nothing; this is the one case
worth flagging as a real gap (nobody can currently index this source).
- **Connector not authorized this session** — report which one and how to
authorize it (claude.ai connector settings / `claude mcp` / `/mcp`),
skip that source, keep going with the rest. Never ask the user for
tokens or callback URLs.
- **Fetching a published index (Step 3) fails or the remote is
unreachable** — report it and fall back to whatever's already cached
locally in `libs/<name>/` from a previous fetch, rather than blocking
the whole run; don't let a transient network issue block a read-only
user from seeing the last-known index.
- **Re-running against an already-indexed source** — refresh existing
entity pages in place (update `tldr`/`last_updated`/`confidence`); never
duplicate a document's page.
- **A previously-indexed document is no longer found at the source**
don't delete its page. Update its `tldr` with a note ("No longer found
at source as of YYYY-MM-DD") and set `retention: low`, so the existing
`ckb-lint` retention sweep archives it naturally on a later pass — no new
archival mechanism needed.
- **A source is very large** — cap what gets fully fetched/summarized in
one run and explicitly report what was skipped (e.g. "42 of ~300
documents summarized this pass — re-run to continue"). Never silently
truncate while implying full coverage.
- **Write access but publishing (Step 7) conflicts with a newer version
someone else already pushed** — this is why Step 3 (fetch) always runs
first, even for write-access users: rebuild on top of the latest fetched
state rather than blindly overwriting it. If a real conflict still shows
up at publish time, report it rather than force-pushing over it.
- **`libs/<name>/` has both real content files and a `source.yaml`** —
ambiguous and out of scope for this skill to resolve; report it and stop
for that folder (this is the same case `ckb-lint`'s external-source
config check flags).
- **`source.yaml` is missing `connector` or `location`** — report it as
malformed and skip that source; don't guess at intent.
---
*Licensed under the Apache License, Version 2.0 — see [LICENSE](../../../LICENSE)
at the repository root, or <http://www.apache.org/licenses/LICENSE-2.0>.*

View file

@ -92,7 +92,7 @@ description. If the page lives in a subdirectory, also add it to that
subdirectory's own `index.md`. Append a log entry to the most specific
applicable log — the subdirectory's `log.md` if it has one, otherwise the
root `wiki/log.md` — following the format in Rule B (`CLAUDE.md`/
`AGENTS.md` §6).
`AGENTS.md` §7).
If this step creates a brand-new `wiki/<topic>/` subdirectory, immediately
create that subdirectory's `index.md` per the Recursive Index & Log
@ -122,7 +122,7 @@ process), skip this reminder — there's nothing to review or sync.
considered, but note in the ingest report that it produced no wiki
change.
- **Conflicting information vs. an existing local wiki page** — this is a
supersession case (Rule via `CLAUDE.md`/`AGENTS.md` §5 lint), not a
supersession case (Rule via `CLAUDE.md`/`AGENTS.md` §6 lint), not a
silent overwrite: update the existing page if the new source is clearly
more current/corroborated, and link `supersedes`/`superseded_by` if an
older version is worth preserving rather than edited in place.

View file

@ -1,6 +1,6 @@
---
name: ckb-lint
description: Health-check the wiki/ — conformance, freshness, confidence decay, retention sweep, supersession detection, orphan detection, graph consistency, index/log consistency, and error-book entries — auto-fixing what it safely can and reporting the rest, then reminding the user to review and sync to origin. Use when the user says "Lint", "health-check the wiki", "check the wiki", or asks for a periodic/scheduled wiki health check. This is the maintenance workflow from CLAUDE.md/AGENTS.md §5, distinct from `ckb-ingest` (which adds new content) and `ckb-sync-changes` (a pure git-level operation with no wiki synthesis at all).
description: Health-check the wiki/ — conformance, freshness, confidence decay, retention sweep, supersession detection, orphan detection, graph consistency, index/log consistency, and error-book entries — auto-fixing what it safely can and reporting the rest, then reminding the user to review and sync to origin. Also runs the same checks against each connector-backed libs/<name>/'s own generated index (see ckb-index-external), plus a source.yaml validity check, without ever touching a git-copy lib or a connector's source.yaml itself. Use when the user says "Lint", "health-check the wiki", "check the wiki", or asks for a periodic/scheduled wiki health check. This is the maintenance workflow from CLAUDE.md/AGENTS.md §6, distinct from `ckb-ingest` (which adds new content) and `ckb-sync-changes` (a pure git-level operation with no wiki synthesis at all).
---
# Lint skill
@ -18,10 +18,17 @@ Implemented as a skill (rather than living inline in
actually invoked — on demand, or on a schedule if the user has set one up.
This skill only ever writes within `wiki/` (including moves into
`wiki/archived/`). It never touches `linked/` or `libs/` — those are
immutable upstream sources of truth — and it never touches this repo's own
git remote; pushing the resulting changes is a separate, explicit step
(see the final step below).
`wiki/archived/`) and, for any connector-backed `libs/<name>/` (one with a
`source.yaml` — see `CLAUDE.md`/`AGENTS.md` §1/§4) where this user has
local `access: write` (see `ckb-index-external`), within that connector's
own agent-owned generated index (`index.md`/`entities/`/`graph/`/`log.md`).
For a connector-backed `libs/<name>/` where this user is read-only (the
default), findings against that index are reported only, never auto-fixed
— the same access gate `ckb-index-external` applies to rebuilding applies
here too. It never touches `linked/`, a git-copy `libs/<name>/`, or a
connector's `source.yaml`/`source.local.yaml` — those stay immutable or
user-owned — and it never touches this repo's own git remote; pushing the
resulting changes is a separate, explicit step (see the final step below).
## Trigger phrases
@ -40,6 +47,16 @@ repo's own commit history with `origin`, no wiki content involved).
Work through each check in order. A malformed page breaks every later
check's ability to reason about it, so conformance comes first.
Checks 1-9 below are written against `wiki/`, but the same checks also run
a second time against each connector-backed `libs/<name>/`'s own generated
index — scoped entirely to that one connector's `index.md`/`entities/`/
`graph/`/`log.md`. Auto-fixes there only happen where this user has local
`access: write` for that source (see `ckb-index-external`); everywhere
read-only, findings are reported but not applied. Findings and fixes there
are logged in that connector's own `log.md`, never in `wiki/log.md`. A
git-copy `libs/<name>/` (no `source.yaml`) and any `source.yaml`/
`source.local.yaml` are never touched by any of these checks.
### 1 — Conformance check
Verify every non-reserved `.md` file under `wiki/` (i.e. excluding
@ -63,7 +80,7 @@ check. Pages that fall below 0.3 confidence get flagged for re-review.
Move `retention: low` pages older than 2× their `freshness_window_days`
into `wiki/archived/`. Never delete — always move, and log the move (see
Rule B in `CLAUDE.md`/`AGENTS.md` §6) with a note explaining why.
Rule B in `CLAUDE.md`/`AGENTS.md` §7) with a note explaining why.
### 5 — Supersession detection
@ -102,6 +119,19 @@ recurring format mismatch, the same kind of orphan appearing again) in
constraint that should prevent recurrence. A one-off, isolated issue
doesn't need an Error Book entry — this is for patterns, not incidents.
### 10 — External source config check
For each `libs/<name>/source.yaml`, verify it has a non-empty `connector`
and `location` — report only, this file is never edited by any skill. If
an `index:` block is present, verify it has a non-empty `store` and
`location` too. Also flag (report only) a `libs/<name>/` that ambiguously
has both real content files and a `source.yaml` — that's a configuration
conflict for the user to resolve, not something to guess at.
Don't flag a missing `libs/<name>/source.local.yaml` as an issue — its
absence is the correct, read-only default (see `ckb-index-external`), not
a gap to report or fix.
### Auto-fix vs. report
Auto-fix what can be done safely and mechanically: broken links, missing
@ -121,6 +151,10 @@ anything yourself here:
> you're happy with them, say 'sync changes' to commit and push them to
> `origin`."
If any connector-backed `libs/<name>/` was also touched, mention it
separately (its own `log.md`, not `wiki/log.md`) rather than folding it
into the same sentence.
If the lint pass made no changes at all (clean bill of health, nothing to
fix or flag), skip this reminder — there's nothing to review or sync.

10
.gitignore vendored
View file

@ -1,5 +1,15 @@
libs/*
!libs/.gitadd
!libs/*/
libs/*/*
!libs/*/source.yaml
!libs/*/index.md
!libs/*/log.md
!libs/*/entities/
!libs/*/graph/
# Per-user local override (e.g. source.local.yaml's access: write) — never shared, stays ignored
# by the libs/*/* catch-all above; listed explicitly for clarity, not because it changes behavior.
libs/*/*.local.yaml
tmp/*
!tmp/.gitadd
outputs/starlight

View file

@ -15,8 +15,16 @@ You possess full autonomy over local directory structure, file naming convention
The root directory contains exactly seven top-level entries. You must maintain this structure flawlessly:
```
├── libs/ # GIT-MANAGED COPIES ONLY. Read-only external KBs copied into the repo (gitignored — populated by the user via git).
│ └── <name>/ # Individual external knowledge base (immutable — never write here).
├── libs/ # Read-only external sources, one of two kinds per <name>/ subfolder:
│ └── <name>/ # - GIT-COPY: a git-managed clone/ZIP unpack, gitignored, fully immutable — never write here.
│ # - CONNECTOR: identified by a user-authored source.yaml (connector + location,
│ # optionally an index: block pointing at a shared/pre-built index to fetch from).
│ # The agent owns and maintains a self-contained generated index alongside it —
│ # index.md/entities/graph/log.md, mirroring wiki/'s own shape but scoped entirely
│ # to this one connector. See §4 EXTERNAL SOURCE INDEXING. source.yaml itself stays
│ # user-only, same as everything in a git-copy lib. Whether *this* user may rebuild
│ # it (vs. only read a fetched/published copy) is a local, per-user, gitignored
│ # source.local.yaml — read-only by default.
├── linked/ # SYMLINKS ONLY. Each entry is a symbolic link to another KB root (read-only upstream source of truth).
│ └── <name>/ # Individual upstream knowledge base (immutable — never write here).
├── outputs/ # MANAGED BY AGENT. Generated artifacts, exports, compiled files produced from the wiki.
@ -43,10 +51,10 @@ When searching for any entity, concept, or file, use the following cascade (firs
1. **Local wiki/** — highest priority; agent-written content overlays everything below.
2. **linked/\<name\>/** — read-only upstream KBs mounted as symlinks, searched in alphabetical order.
3. **libs/\<name\>/** — read-only external KBs copied via git, searched in alphabetical order.
3. **libs/\<name\>/** — read-only external sources, searched in alphabetical order. For a git-copy lib this is its cloned files; for a connector-backed lib (one with a `source.yaml`) this layer's content *is* the agent-generated index (`index.md`/`entities/`/`graph/`) built by the `ckb-index-external` skill, not raw copied files — see §4.
4. If no match is found anywhere, treat the entity as unknown.
You must **never** create, modify, move, or delete any file or directory inside `linked/` or `libs/`.
You must **never** create, modify, move, or delete any file or directory inside `linked/` or a git-copy `libs/<name>/`. The one exception is a connector-backed `libs/<name>/`'s own generated index, which the agent owns and maintains exactly like `wiki/` — see Rule A in §7.
### Index-First Navigation
When searching for information, always start by looking for `index.md` files.
@ -65,6 +73,7 @@ Index-First Navigation only works if subdirectory indexes actually exist. Mainta
- Every `wiki/` subdirectory that groups multiple pages (`entities/`, `graph/`, and any future topic folder) must contain its own `index.md`. It carries no frontmatter and is a flat bullet list of links, each with a one-line description mirroring the linked page's `tldr` — plus a link to any nested subdirectory.
- A subdirectory may also keep its own `log.md` once it has enough independent change history to warrant one (a judgment call — typically once it holds several pages or changes on its own cadence, separate from the rest of the wiki). Entries follow the same reverse-chronological format as Rule B.
- The root `wiki/log.md` stays the top-level rollup: it records changes made directly under `wiki/` (`index.md`, `overview.md`, `error-book.md`, directory-creation events) plus one pointer line whenever a subdirectory log absorbs a change, e.g. `- See wiki/entities/log.md for entity-page changes on this date.` Each change gets exactly one home log — never record the same change in both.
- The same convention applies verbatim inside a connector-backed `libs/<name>/` (§4) — its generated `index.md`/`entities/index.md`/`graph/index.md`/`log.md` mirror this pattern exactly, scoped entirely to that one connector. Its `log.md` is independent of `wiki/log.md` — never record a connector-indexing change in both.
### Lazy-Loading with "Use When" Triggers
The `wiki/index.md` is a routing table. Each entry has a **Use when** column
@ -119,7 +128,13 @@ When the user says "Ingest", "Sync the wiki", or "Update the Wiki" (for syncing
---
## 4. QUERY WORKFLOW
## 4. EXTERNAL SOURCE INDEXING (TRIGGERED ON DEMAND)
When the user says "Index external sources" (or "index libs", "refresh the external index"), run the **ckb-index-external** Claude Code Skill — see `.agents/skills/ckb-index-external/SKILL.md` — rather than following inline steps here, so the full procedure only loads into context when actually invoked. It walks every connector-backed `libs/<name>/` (one with a `source.yaml` — see §1), fetches a shared/pre-built index if `source.yaml` declares one (`index.store`/`index.location` — git or a shared resource), and — only if this user has local `access: write` in `libs/<name>/source.local.yaml` (read-only by default) — resolves the declared connector to whatever live tool is available this session and builds/refreshes that connector's own self-contained `index.md`/`entities/`/`graph/`/`log.md`, publishing it back to the shared store if one is configured. This never touches `wiki/`, never touches `source.yaml`, and never touches a git-copy lib.
---
## 5. QUERY WORKFLOW
When answering a question or researching a topic:
@ -127,20 +142,24 @@ When answering a question or researching a topic:
2. **Read TLDRs** — for any matched page, read its `tldr:` frontmatter first. If it answers the query, stop.
3. **Load full pages** — only if the TLDR was insufficient.
4. **Walk the graph** — if the entity has relationships in `wiki/graph/edges.json`, follow them to discover connected pages (e.g. "what depends on X?").
5. **Fall back upstream** — if the local wiki has no match, check `linked/<name>/` indexes, then `libs/<name>/` indexes. Apply cascade priority throughout.
5. **Fall back upstream** — if the local wiki has no match, check `linked/<name>/` indexes, then `libs/<name>/` indexes (for a connector-backed lib, that means its generated `entities/`/`index.md`, not the live source directly — if it's not there yet, suggest running "index external sources" rather than fetching the live source ad hoc). Apply cascade priority throughout.
---
## 5. MAINTENANCE WORKFLOW (LINT)
## 6. MAINTENANCE WORKFLOW (LINT)
Periodically (or when asked to "Lint"), run the **ckb-lint** Claude Code Skill — see `.agents/skills/ckb-lint/SKILL.md` — rather than following inline steps here, so the full checklist (conformance, freshness, confidence decay, retention sweep, supersession detection, orphan detection, graph consistency, index/log consistency, error-book entries, auto-fix vs. report, then a reminder to review and sync) only loads into context when actually invoked.
---
## 6. COMPLIANCE & LOGGING RULES (NON-NEGOTIABLE)
## 7. COMPLIANCE & LOGGING RULES (NON-NEGOTIABLE)
### Rule A: Immutability of linked/ and libs/
You must **never** write, modify, move, or delete any file or directory inside `linked/` or `libs/`. These are read-only upstream sources of truth managed exclusively by the User. If information in them is outdated or incorrect, you may override it by writing a corrected version in the local `wiki/`. The local version will take priority in the cascade lookup.
You must **never** write, modify, move, or delete any file or directory inside `linked/` or a git-copy `libs/<name>/`. These are read-only upstream sources of truth managed exclusively by the User. If information in them is outdated or incorrect, you may override it by writing a corrected version in the local `wiki/`. The local version will take priority in the cascade lookup.
**Exception — connector-backed `libs/<name>/`:** identified by the presence of a `source.yaml` (see §1). Its `source.yaml` is user-authored and stays just as untouchable as anything else here. But everything else in that folder — `index.md`, `entities/`, `graph/`, `log.md` — is a generated index the agent owns and maintains exactly as it would `wiki/`, built and refreshed by the `ckb-index-external` skill (§4). This exception applies only to a `libs/<name>/` that has a `source.yaml`; a plain git-copy lib has no such carve-out.
Within that exception, two things the agent may always do regardless of this user's access level: fetch a shared/pre-built index down into `libs/<name>/` if `source.yaml` declares one, and read whatever's cached there. Actually rebuilding it from the live connector — and publishing that rebuild back to a shared store — is gated by a separate, local, per-user `libs/<name>/source.local.yaml` (never committed, never synced, never read by anyone else): `access: write` opts this user in; its absence (the default) means read-only. Unlike `source.yaml`, the agent *may* create or edit `source.local.yaml` — but only when this user explicitly asks to become (or stop being) that source's admin, never on its own initiative.
### Rule B: The Wiki Change Log (`wiki/log.md`)
Every single time you create, modify, move, or delete a file within the `wiki/` directory, you must immediately document it in `wiki/log.md` before proceeding.
@ -171,6 +190,7 @@ After every conversational turn where you take any action (read, write, search,
Follow these event-driven behaviors:
- **On new source in inbox** — on the next ingest, auto-process: extract entities, update graph, update index, write to log.
- **On new or changed `libs/<name>/source.yaml`** — on the next "index external sources" run, process it: resolve the connector, enumerate documents, build/refresh that connector's own `index.md`/`entities/`/`graph/`/`log.md`.
- **On session start** — read `wiki/index.md` and the latest `workload/` summary to load relevant context. Also check for unsynchronized changes (`git status` — uncommitted local changes, or the local branch ahead/behind its remote-tracking ref) and, if any are found, tell the user and suggest running the `ckb-sync-changes` skill before proceeding. This is a cheap, read-only check (no `git fetch`) — a heads-up, not a substitute for actually running that skill.
- **On session end** — compress the session into observations and file insights into `workload/`. Also re-run the same unsynchronized-changes check as at session start — the session's own work may have just created new local changes — and suggest `ckb-sync-changes` if anything is now pending.
- **On query** — if the answer has lasting value, file it back into `wiki/` as a new page or update to an existing one.

135
MANUAL.md
View file

@ -55,7 +55,7 @@ If the target folder already looks like a KB (it has a `wiki/` or
### Building on top of someone else's wiki
A Cascade KB can sit on top of one or more *upstream* knowledge bases,
which stay entirely read-only. There are two ways to mount one:
which stay entirely read-only. There are three ways to mount one:
- **Symlink** (another KB on your own machine, or one you maintain
elsewhere and want live-linked):
@ -73,13 +73,48 @@ which stay entirely read-only. There are two ways to mount one:
frozen, read-only copy either way; the only difference is you won't be
able to `git pull` it later to refresh — to update it, just re-download
the ZIP and unpack it over the old contents.
- **Connector** (a live external source you *don't* want a full local
copy of — a SharePoint folder, a Google Drive folder, or another
connected source): create `libs/<name>/source.yaml` yourself:
```yaml
connector: sharepoint
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Finance team's shared reports folder"
```
then say "index external sources." The agent reads the config, connects
to whatever's available that session (a connected Microsoft 365/Google
Drive tool, or a plain URL fetch), and builds a short index of what it
finds — one entry per document — inside that same `libs/<name>/` folder.
See [§6](#6-use-cases) for a worked example and what the result looks
like.
Two things worth knowing up front about a connector source:
- **You don't have to build the index yourself.** `source.yaml` can add
an `index:` block pointing at an already-built index — a git repo, or
a shared resource — so you just fetch what someone else already
indexed instead of scanning the live source yourself. Every run checks
that location first: if there's already an index there, you get it; if
there isn't yet (the normal state before anyone's run this with write
access), that's not an error — whoever has write access is the one
whose next run creates and publishes it there for the first time.
- **Building/refreshing is opt-in per person, per source.** By default,
everyone is read-only for a connector source — nobody's agent will
scan the live connector on their behalf unless they've explicitly said
so. Say "make me the admin for `<source>`" to opt yourself in (this
writes a local, personal `libs/<name>/source.local.yaml` — never
committed, never seen by anyone else). This is deliberate: it lets one
or two people maintain a source for the whole team instead of everyone
redundantly re-scanning it.
Either way, once it's mounted, just ask questions normally — the agent
checks your local `wiki/` first, then walks `linked/`, then `libs/`, and
uses whichever has the answer. You never edit files inside `linked/` or
`libs/` directly; if something there is wrong or outdated, you fix it by
writing the corrected version into your own local `wiki/`, which always
wins.
uses whichever has the answer. You never edit files inside `linked/` or a
git-copy `libs/<name>/` directly; if something there is wrong or outdated,
you fix it by writing the corrected version into your own local `wiki/`,
which always wins. (A connector-backed `libs/<name>/` is the one place the
agent *does* write on your behalf — see [§6](#6-use-cases) — but only its
generated index, and only the build/refresh part if you're that source's
admin; `source.yaml` itself always stays yours to edit, never the agent's.)
---
@ -396,6 +431,90 @@ Covered in full in [§2](#2-adding-knowledge) — the short version: drop it
in `raw/inbox/` and say "Ingest," or just tell the agent in chat if it's
small enough to state directly.
### Index an external source
Say:
> "Index external sources." (or "index libs," "refresh the external
> index")
This walks every `libs/<name>/` that has a `source.yaml` (see
[§1](#building-on-top-of-someone-elses-wiki)) and builds a short index of
what it finds — one entry per document, plus an overview page — entirely
inside that same `libs/<name>/` folder. Nothing under `wiki/` is touched.
Example:
> *You create `libs/finance-reports/source.yaml`:*
> ```yaml
> connector: sharepoint
> location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
> description: "Finance team's shared reports folder"
> ```
> *then say "Index external sources."*
>
> The agent connects using whatever's available that session (a connected
> Microsoft 365 tool, in this case), lists the documents in that folder,
> reads enough of each to write a short summary, and creates
> `libs/finance-reports/index.md` (an overview of the source) plus one
> page per document under `libs/finance-reports/entities/`, cross-linked
> via `libs/finance-reports/graph/edges.json`. It logs everything in
> `libs/finance-reports/log.md` — a log that's entirely separate from
> `wiki/log.md`, since this index is scoped to that one connector rather
> than blended into your main wiki. It finishes by reminding you to review
> the result and say "sync changes" once you're happy with it.
If a connector needs authorizing (e.g. a SharePoint or Google Drive
connection that isn't set up yet), the agent tells you which one and where
to authorize it, then continues with any other configured sources rather
than stopping the whole run. Run "index external sources" again any time
the source has changed — it refreshes existing entries in place rather
than duplicating them, and never deletes a page for a document that's
disappeared from the source (it flags it instead, so a later "Lint" pass
archives it naturally). Implemented by the `ckb-index-external` skill —
`.agents/skills/ckb-index-external/SKILL.md`.
**Who's allowed to build it, and where it's shared.** By default, nobody
has write access to a connector source until they say so — this keeps a
team of, say, ten people from all redundantly scanning the same SharePoint
folder. Say:
> "Make me the admin for finance-reports."
This writes a personal `libs/finance-reports/source.local.yaml` with
`access: write` — never committed, never seen by your teammates. Anyone
without that file is read-only for that source: if they say "index
external sources," the agent won't touch the live connector on their
behalf at all — it just reports what's already indexed (or tells them
plainly that nothing has been indexed yet and who to ask).
If the finance team wants everyone reading the *same* index rather than
each maintaining their own local copy of it inside their own KB, the admin
adds an `index:` block to the shared `source.yaml`:
```yaml
index:
store: git
location: "https://github.com/finance-team/index-cache.git"
# ref: main — optional: pin a branch, tag, or subpath within that store
```
The very first time anyone runs "index external sources" after that block
is added, `https://github.com/finance-team/index-cache.git` is empty —
that's expected, not an error. Every run checks it first: read-only users
just see "nothing published yet, ask the admin"; the admin's run is what
actually creates it there, since a write-access run always rebuilds from
the live connector and pushes the result to that location, whether or not
anything was there before. From then on, whenever *anyone* says "index
external sources," the agent first fetches whatever's already published
there — read-only users stop right there; the admin also rebuilds from the
live connector and pushes the refreshed index back to that same location,
so the next person's fetch picks it up. Leave the `index:` block out
entirely (the simplest setup, and the right default for a single small
team) and the index just lives
directly inside `libs/finance-reports/` in this KB's own repo, shared the
normal way via "sync changes" — exactly like the plain example above.
---
## 7. What's agent-generated vs. what you can edit
@ -413,7 +532,10 @@ graph stay in sync with what you changed.
| `raw/inbox/`, loose files in `raw/` | **You, only** | The agent only reads, archives, and moves things here — it never originates content in `raw/` itself. |
| `raw/archive/<date>/` | Agent | Auto-filed copy of what you dropped in `raw/inbox/`, organised by ingestion date. Don't hand-file here — let Ingest do it, so the date and pairing with the log entry stay accurate. |
| `linked/<name>/` | **You** (you create the symlink) | Points at another KB's real files, which live and get edited *in that other repo* — never here. The agent must never write inside `linked/`. |
| `libs/<name>/` | **You** (you `git clone`) | A frozen copy of an external KB. Update it by re-pulling that repo yourself, not by hand-editing files here. The agent must never write inside `libs/`. |
| `libs/<name>/` (git-copy, no `source.yaml`) | **You** (you `git clone`) | A frozen copy of an external KB. Update it by re-pulling that repo yourself, not by hand-editing files here. The agent must never write inside it. |
| `libs/<name>/source.yaml` (connector) | **You, only** | Declares the connector, location, and optionally where a shared/pre-built index lives (`index:`). The agent reads it but never writes it — same as anything else upstream. |
| `libs/<name>/source.local.yaml` (connector) | **You** (or the agent, only when you explicitly ask to become/stop being that source's admin) | Per-person, per-machine `access: write`/`read` setting — never committed, never seen by anyone else. Absent = read-only, the default. |
| `libs/<name>/{index.md,entities/,graph/,log.md}` (connector) | Agent-generated, **you can freely edit** | The agent's own index of that one connector's source, built/refreshed by "Index external sources" — but only if you have `access: write` locally; read-only users just get a fetched copy. Structurally the same deal as the main `wiki/` row below — feel free to correct an entry by hand, then run "Lint" (it now also checks connector-backed indexes, respecting the same read/write split). Scoped entirely to that connector; never blended into `wiki/`. |
| `wiki/` (pages, `index.md`, `overview.md`, `log.md`, `error-book.md`, `entities/`, `graph/`) | Agent-generated, **you can freely edit** | This is the one place the agent both writes and expects you might too. Feel free to correct a page by hand — just keep the frontmatter fields intact (or update `last_updated`), and run Lint afterward if you touched something the index/graph/log reference. |
| `outputs/okf/`, `outputs/starlight/` | Agent, **fully regenerated** | Don't hand-edit — these are gitignored build artifacts, silently overwritten the next time you export. If something's wrong, fix the wiki page it came from and re-export. |
| `outputs/teaching/<topic>/` | Agent, semi-persistent state | `plan.md`/`progress.md` the teaching skill reads and writes across sessions. You can look at them any time; hand-editing is possible but may confuse "what's next" tracking — safer to tell the agent what you want changed and let it update the files. |
@ -440,4 +562,5 @@ graph stay in sync with what you changed.
| "Export the wiki as OKF" | Machine-readable export at `outputs/okf/` | `ckb-export-okf` |
| "Export the wiki to Starlight" | Human-readable docs site at `outputs/starlight/` | `ckb-export-starlight` |
| "Upgrade the wiki" / "Check for a newer template version" | Checks template + wiki schema versions against the canonical repo, upgrades what you accept | `ckb-upgrade` |
| "Index external sources" / "Index libs" | Builds/refreshes a self-contained index for each connector-backed `libs/<name>/` | `ckb-index-external` |
| Just ask a question | Answers from the wiki, using the index/TLDR/graph cascade | — (core query workflow) |

View file

@ -56,7 +56,7 @@ nadpisze po cichu istniejącej bazy wiedzy.
### Budowanie na bazie cudzej wiki
Cascade KB może opierać się na jednej lub wielu *nadrzędnych* bazach
wiedzy, które pozostają całkowicie tylko do odczytu. Są dwa sposoby ich
wiedzy, które pozostają całkowicie tylko do odczytu. Są trzy sposoby ich
podpięcia:
- **Dowiązanie symboliczne** (inna KB na twojej maszynie, lub taka, którą
@ -76,13 +76,52 @@ podpięcia:
brak możliwości późniejszego `git pull`, żeby ją odświeżyć — żeby
zaktualizować, po prostu pobierz ZIP ponownie i rozpakuj go na starą
zawartość.
- **Konektor** (żywe zewnętrzne źródło, którego *nie* chcesz mieć w pełnej
lokalnej kopii — folder SharePoint, folder Google Drive albo inne
podłączone źródło): sam utwórz `libs/<name>/source.yaml`:
```yaml
connector: sharepoint
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Wspólny folder raportów zespołu finansowego"
```
a potem powiedz „index external sources". Agent czyta konfigurację,
łączy się z tym, co jest dostępne w danej sesji (podłączonym narzędziem
Microsoft 365/Google Drive albo zwykłym pobraniem URL), i buduje krótki
indeks tego, co znajdzie — po jednym wpisie na dokument — wewnątrz tego
samego folderu `libs/<name>/`. Zobacz [§6](#6-przykłady-użycia) po
omówiony przykład i to, jak wygląda wynik.
Dwie rzeczy warto wiedzieć z góry o źródle typu konektor:
- **Nie musisz sam budować indeksu.** `source.yaml` może dodać blok
`index:` wskazujący na już zbudowany indeks — repozytorium git albo
zasób współdzielony — dzięki czemu po prostu pobierasz to, co ktoś
inny już zaindeksował, zamiast samodzielnie skanować żywe źródło.
Każde uruchomienie najpierw sprawdza tę lokalizację: jeśli indeks już
tam jest, dostajesz go; jeśli go tam jeszcze nie ma (normalny stan,
zanim ktokolwiek z dostępem do zapisu to uruchomił), to nie błąd —
ten, kto ma dostęp do zapisu, tworzy go tam i publikuje przy swoim
kolejnym uruchomieniu.
- **Budowanie/odświeżanie jest opcjonalne, per osoba, per źródło.**
Domyślnie każdy jest tylko-do-odczytu dla źródła typu konektor —
agent nikogo nie przeskanuje żywego konektora w jego imieniu, jeśli
wyraźnie tego nie zadeklarował. Powiedz „make me the admin for
`<source>`", żeby się na to zapisać (tworzy to lokalny, osobisty plik
`libs/<name>/source.local.yaml` — nigdy niecommitowany, nigdy
niewidoczny dla współpracowników). To celowe: pozwala jednej lub dwóm
osobom utrzymywać źródło dla całego zespołu, zamiast żeby każdy
redundantnie je skanował.
Niezależnie od sposobu, po podpięciu wystarczy normalnie zadawać pytania —
agent sprawdza najpierw twoją lokalną `wiki/`, potem przechodzi przez
`linked/`, potem `libs/`, i korzysta z tego, co ma odpowiedź. Nigdy nie
edytujesz plików wewnątrz `linked/` ani `libs/` bezpośrednio; jeśli coś tam
jest błędne lub nieaktualne, poprawiasz to, zapisując poprawioną wersję we
własnej lokalnej `wiki/`, która zawsze wygrywa.
edytujesz plików wewnątrz `linked/` ani kopii git w `libs/<name>/`
bezpośrednio; jeśli coś tam jest błędne lub nieaktualne, poprawiasz to,
zapisując poprawioną wersję we własnej lokalnej `wiki/`, która zawsze
wygrywa. (`libs/<name>/` oparty na konektorze to jedyne miejsce, gdzie
agent *sam* zapisuje w twoim imieniu — zobacz [§6](#6-przykłady-użycia) —
ale tylko swój generowany indeks, i tylko część budowania/odświeżania,
jeśli jesteś administratorem tego źródła; `source.yaml` zawsze pozostaje
twój do edycji, nigdy agenta.)
---
@ -413,6 +452,97 @@ W pełni opisane w [§2](#2-dodawanie-wiedzy) — w skrócie: wrzuć materiał d
`raw/inbox/` i powiedz „Ingest”, albo po prostu powiedz agentowi w
rozmowie, jeśli to wystarczająco krótkie, żeby podać wprost.
### Indeksowanie zewnętrznego źródła
Powiedz:
> „Index external sources.” (lub „index libs”, „refresh the external
> index”)
To przeszukuje każdy `libs/<name>/`, który ma `source.yaml` (zobacz
[§1](#budowanie-na-bazie-cudzej-wiki)), i buduje krótki indeks tego, co
znajdzie — po jednym wpisie na dokument, plus stronę przeglądową — w
całości wewnątrz tego samego folderu `libs/<name>/`. Nic pod `wiki/` nie
jest dotykane.
Przykład:
> *Tworzysz `libs/finance-reports/source.yaml`:*
> ```yaml
> connector: sharepoint
> location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
> description: "Wspólny folder raportów zespołu finansowego"
> ```
> *potem mówisz „Index external sources.”*
>
> Agent łączy się, korzystając z tego, co jest dostępne w danej sesji (w
> tym przypadku podłączonego narzędzia Microsoft 365), listuje dokumenty w
> tym folderze, czyta wystarczająco dużo z każdego, aby napisać krótkie
> podsumowanie, i tworzy `libs/finance-reports/index.md` (przegląd źródła)
> plus jedną stronę na dokument w `libs/finance-reports/entities/`,
> połączone krzyżowo przez `libs/finance-reports/graph/edges.json`. Loguje
> wszystko w `libs/finance-reports/log.md` — dzienniku całkowicie
> odrębnym od `wiki/log.md`, ponieważ ten indeks jest ograniczony do
> jednego konektora, a nie wmieszany w twoją główną wiki. Na koniec
> przypomina o przejrzeniu wyniku i powiedzeniu „sync changes”, gdy
> będziesz zadowolony.
Jeśli konektor wymaga autoryzacji (np. połączenie z SharePoint lub Google
Drive, które nie jest jeszcze skonfigurowane), agent mówi, który to i gdzie
go autoryzować, a potem kontynuuje z innymi skonfigurowanymi źródłami,
zamiast zatrzymywać cały przebieg. Uruchom „index external sources”
ponownie w każdej chwili, gdy źródło się zmieni — odświeża istniejące
wpisy w miejscu, zamiast je duplikować, i nigdy nie usuwa strony dla
dokumentu, który zniknął ze źródła (zamiast tego oflagowuje ją, żeby
kolejny przebieg „Lint” zarchiwizował ją naturalnie). Zaimplementowane
przez skill `ckb-index-external`
`.agents/skills/ckb-index-external/SKILL.md`.
**Kto może go budować i gdzie jest współdzielony.** Domyślnie nikt nie ma
dostępu do zapisu w źródle typu konektor, dopóki tego nie zadeklaruje — to
chroni zespół dziesięciu osób przed redundantnym skanowaniem tego samego
folderu SharePoint. Powiedz:
> „Make me the admin for finance-reports.”
To zapisuje osobisty `libs/finance-reports/source.local.yaml` z
`access: write` — nigdy niecommitowany, nigdy niewidoczny dla
współpracowników. Każdy bez tego pliku jest tylko-do-odczytu dla tego
źródła: jeśli powie „index external sources”, agent w jego imieniu w
ogóle nie dotknie żywego konektora — po prostu zgłosi, co już
zaindeksowano (albo powie wprost, że nic jeszcze nie zaindeksowano i kogo
o to zapytać).
Jeśli zespół finansowy chce, żeby wszyscy czytali *ten sam* indeks, a nie
każdy utrzymywał własną lokalną kopię w swojej własnej KB, administrator
dodaje blok `index:` do współdzielonego `source.yaml`:
```yaml
index:
store: git
location: "https://github.com/finance-team/index-cache.git"
# ref: main — opcjonalnie: przypina branch, tag albo podścieżkę w tym miejscu
```
Za pierwszym razem, gdy ktokolwiek uruchomi „index external sources” po
dodaniu tego bloku, `https://github.com/finance-team/index-cache.git` jest
puste — to oczekiwane, nie błąd. Każde uruchomienie sprawdza je najpierw:
użytkownicy tylko-do-odczytu zobaczą po prostu „nic jeszcze nie
opublikowano, zapytaj administratora”; to uruchomienie administratora
faktycznie je tworzy, ponieważ uruchomienie z dostępem do zapisu zawsze
przebudowuje indeks z żywego konektora i wypycha wynik do tej lokalizacji,
niezależnie od tego, czy coś tam wcześniej było. Od tego momentu, kiedy
*ktokolwiek* powie „index external sources”, agent najpierw
pobiera to, co już zostało opublikowane — użytkownicy tylko-do-odczytu
zatrzymują się w tym miejscu; administrator dodatkowo przebudowuje indeks
z żywego konektora i wypycha odświeżoną wersję do tej samej lokalizacji,
żeby kolejne pobranie innej osoby ją uwzględniło. Pomiń blok `index:` w
ogóle (najprostsza konfiguracja, właściwy domyślny wybór dla jednego
małego zespołu), a indeks po prostu żyje bezpośrednio wewnątrz
`libs/finance-reports/` we własnym repozytorium tej KB, współdzielony w
normalny sposób przez „sync changes” — zupełnie jak w prostym przykładzie
powyżej.
---
## 7. Co jest generowane przez agenta, a co możesz edytować
@ -431,7 +561,10 @@ zmieniłeś.
| `raw/inbox/`, luźne pliki w `raw/` | **Tylko ty** | Agent tylko czyta, archiwizuje i przenosi rzeczy tutaj — nigdy nie tworzy treści w `raw/` sam. |
| `raw/archive/<data>/` | Agent | Automatycznie zarchiwizowana kopia tego, co wrzuciłeś do `raw/inbox/`, uporządkowana według daty ingestu. Nie umieszczaj tu plików ręcznie — pozwól, żeby zrobił to Ingest, tak by data i powiązanie z wpisem w dzienniku były poprawne. |
| `linked/<name>/` | **Ty** (tworzysz dowiązanie symboliczne) | Wskazuje na rzeczywiste pliki innej KB, które żyją i są edytowane *w tamtym repozytorium* — nigdy tutaj. Agent nigdy nie może zapisywać wewnątrz `linked/`. |
| `libs/<name>/` | **Ty** (robisz `git clone`) | Zamrożona kopia zewnętrznej KB. Aktualizujesz ją, ponownie pobierając to repozytorium samodzielnie, a nie ręcznie edytując pliki tutaj. Agent nigdy nie może zapisywać wewnątrz `libs/`. |
| `libs/<name>/` (kopia git, bez `source.yaml`) | **Ty** (robisz `git clone`) | Zamrożona kopia zewnętrznej KB. Aktualizujesz ją, ponownie pobierając to repozytorium samodzielnie, a nie ręcznie edytując pliki tutaj. Agent nigdy nie może zapisywać wewnątrz niej. |
| `libs/<name>/source.yaml` (konektor) | **Tylko ty** | Deklaruje konektor, lokalizację i opcjonalnie gdzie znajduje się współdzielony/wcześniej zbudowany indeks (`index:`). Agent go czyta, ale nigdy nie zapisuje — tak jak wszystko inne nadrzędne. |
| `libs/<name>/source.local.yaml` (konektor) | **Ty** (albo agent, tylko gdy wyraźnie poprosisz o zostanie/przestanie bycia administratorem tego źródła) | Osobiste, per-komputer ustawienie `access: write`/`read` — nigdy niecommitowane, nigdy niewidoczne dla innych. Brak = tylko do odczytu, domyślnie. |
| `libs/<name>/{index.md,entities/,graph/,log.md}` (konektor) | Generowane przez agenta, **możesz swobodnie edytować** | Własny indeks agenta dla tego jednego źródła konektora, budowany/odświeżany przez „Index external sources” — ale tylko jeśli masz lokalnie `access: write`; użytkownicy tylko-do-odczytu dostają po prostu pobraną kopię. Strukturalnie ta sama zasada jak przy wierszu `wiki/` poniżej — śmiało popraw wpis ręcznie, a potem uruchom „Lint” (teraz sprawdza też indeksy oparte na konektorach, respektując ten sam podział odczyt/zapis). Ograniczone wyłącznie do tego konektora; nigdy nie wmieszane w `wiki/`. |
| `wiki/` (strony, `index.md`, `overview.md`, `log.md`, `error-book.md`, `entities/`, `graph/`) | Generowane przez agenta, **możesz swobodnie edytować** | To jedyne miejsce, w którym zarówno agent zapisuje, jak i spodziewa się, że ty też możesz. Śmiało popraw stronę ręcznie — zachowaj tylko pola frontmatteru (lub zaktualizuj `last_updated`) i uruchom potem Lint, jeśli dotknąłeś czegoś, do czego odwołuje się indeks/graf/dziennik. |
| `outputs/okf/`, `outputs/starlight/` | Agent, **w pełni regenerowane** | Nie edytuj ręcznie — to zignorowane przez git artefakty budowania, cicho nadpisywane przy każdym kolejnym eksporcie. Jeśli coś jest nie tak, popraw stronę wiki, z której to pochodzi, i wyeksportuj ponownie. |
| `outputs/teaching/<topic>/` | Agent, stan półtrwały | `plan.md`/`progress.md`, które skill do nauczania czyta i zapisuje między sesjami. Możesz je oglądać kiedy chcesz; ręczna edycja jest możliwa, ale może pomieszać śledzenie „co dalej” — bezpieczniej powiedzieć agentowi, co chcesz zmienić, i pozwolić mu zaktualizować pliki. |
@ -458,4 +591,5 @@ zmieniłeś.
| „Export the wiki as OKF” | Eksport maszynowy w `outputs/okf/` | `ckb-export-okf` |
| „Export the wiki to Starlight” | Czytelna dla człowieka strona dokumentacji w `outputs/starlight/` | `ckb-export-starlight` |
| „Upgrade the wiki” / „Check for a newer template version” | Sprawdza wersje szablonu i schematu wiki względem kanonicznego repozytorium, aktualizuje to, co zaakceptujesz | `ckb-upgrade` |
| „Index external sources” / „Index libs” | Buduje/odświeża samodzielny indeks dla każdego `libs/<name>/` opartego na konektorze | `ckb-index-external` |
| Po prostu zadaj pytanie | Odpowiedź z wiki, przy użyciu kaskady indeks/TLDR/graf | — (podstawowy przepływ zapytań) |

View file

@ -19,7 +19,8 @@ worked examples for every use case — see [MANUAL.md](MANUAL.md)
## Directory Structure
```
├── libs/ # Read-only external KBs copied via git (gitignored)
├── libs/ # Read-only external sources — git-copy clones (gitignored) OR
│ # connector configs (source.yaml) with a self-contained generated index
├── linked/ # Read-only upstream KBs mounted as symlinks
├── outputs/ # Generated artifacts, exports, compiled files
├── raw/ # User-provided source material
@ -46,11 +47,14 @@ When searching, layers are checked in order — first match wins:
wiki/ (highest) ← agent writes here, always wins
linked/A/ (medium) ← symlinked upstream KBs
linked/B/ (low) ← symlinked upstream KBs
libs/A/ (lowest) ← git-managed external KB copies
libs/A/ (lowest) ← git-managed external KB copies, or a connector's own generated index
```
The agent never writes to `linked/` or `libs/`. To correct upstream content,
write the right version in `wiki/` — it takes precedence automatically.
The agent never writes to `linked/` or a git-copy `libs/<name>/`. To correct
upstream content, write the right version in `wiki/` — it takes precedence
automatically. The one exception is a connector-backed `libs/<name>/` (see
"External Source Connectors & Indexing" below) — the agent owns and
maintains its generated index exactly as it would `wiki/`.
---
@ -124,6 +128,58 @@ full scan. A subdirectory can also keep its own `log.md` once it has enough
independent history; `wiki/log.md` stays the root-level rollup and never
duplicates a change a subdirectory log already recorded.
### External Source Connectors & Indexing (on demand)
A `libs/<name>/` folder supports a second population mode alongside the
existing git-copy one: a small user-authored `libs/<name>/source.yaml`
declaring a *live* external source — a SharePoint folder, a Google Drive
folder, a plain URL, or another connector — that you don't want to fully
mirror locally:
```yaml
connector: sharepoint
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Finance team's shared reports folder"
```
Say "index external sources" and the agent walks it, resolving `connector`
to whatever live tool is available that session (a connected Microsoft
365/Google Drive MCP tool, or `WebFetch` for a plain URL), and builds a
self-contained generated index inside that same `libs/<name>/`
`index.md`/`entities/`/`graph/`/`log.md`, mirroring `wiki/`'s own shape via
the Recursive Index & Log Convention above, but scoped entirely to that one
connector. This is a deliberate design choice: the index is **not** blended
into the main `wiki/entities/`/`wiki/graph/edges.json` — it stays separated
at the `libs/` cascade layer, the same way a git-cloned KB's own files
already are. `source.yaml` itself stays user-only, never written by the
agent.
Two refinements on top of that:
- **Shared, pre-built indexes.** `source.yaml` can add an optional `index:`
block declaring *where the already-built index lives* — a git repo, or a
shared resource such as a network path or another connector-reachable
location:
```yaml
index:
store: git # git | shared
location: "https://github.com/org/finance-index-cache.git"
ref: main # optional — branch, tag, or subpath hint within that store
```
Every run checks that location: if it already has an index, fetch it —
most people just read what's already there instead of building it
themselves. If it doesn't yet, that's the normal first-time state, not an
error: a write-access user's very next run is what creates and publishes
it there, with no separate "initialize" step.
- **Read vs. write, per user, per source.** Whether *this* user can
actually rebuild an index (versus only read a fetched/published one) is a
separate, local, gitignored `libs/<name>/source.local.yaml` — read-only
by default. Setting `access: write` there opts a given machine/user in as
that source's admin, so a team can designate one or two people to
maintain a source while everyone else just reads the result — no
redundant rebuilding, no need for every user to have their own connector
authorization.
Implemented as a Claude Code Skill — see
`.agents/skills/ckb-index-external/SKILL.md`.
### Dual-Linking (Wikilinks + Markdown)
Every cross-reference uses both `[[Wikilinks]]` (Obsidian-compatible) and
standard `[markdown](path.md)` links. Works in Obsidian graph view, GitHub
@ -283,6 +339,9 @@ advancing. Never writes to `wiki/`. See
ln -s /path/to/other-kb ./linked/my-upstream
git clone https://github.com/org/external-kb ./libs/external-kb
```
Or, for a live source you don't want to fully mirror, drop a
`libs/<name>/source.yaml` instead (see "External Source Connectors &
Indexing" above) and say "index external sources."
2. **Drop raw material** into `raw/inbox/` (notes, links, articles).
@ -313,13 +372,25 @@ copy to keep in sync.
## Tips
- Upstream KBs (`linked/` and `libs/`) are **never modified** by agents.
- Upstream KBs (`linked/` and git-copy `libs/`) are **never modified** by
agents. A connector-backed `libs/<name>/` (one with a `source.yaml`) is
the one exception — the agent owns and maintains its generated index,
but only for a user who's opted themselves into `access: write` locally
(see the next point); everyone else's copy stays read-only.
- To correct upstream content, write the correct version in `wiki/` — it wins.
- Use `raw/inbox/` for anything unprocessed; the agent clears it on ingest.
- The `wiki/index.md` routing table is the most important file — keep it current.
- Confidence, quality, and freshness let you trust the right content and
flag the rest for review.
- The `tmp/` and `libs/` directories are gitignored. `outputs/` itself is
- The `tmp/` directory is gitignored, and so is most of `libs/` — but not
all of it: a git-copy `libs/<name>/`'s cloned content stays gitignored as
before, while a connector-backed `libs/<name>/`'s `source.yaml` and its
generated `index.md`/`entities/`/`graph/`/`log.md` are tracked, since
they're synthesized knowledge worth sharing via "sync changes," not a
disposable build artifact. `libs/<name>/source.local.yaml` (per-user
read/write setting) is the one exception that stays gitignored right
alongside them — it's personal machine state, never meant to sync.
`outputs/` itself is
tracked, but its regenerated build subdirectories, `outputs/okf/` and
`outputs/starlight/`, are gitignored — each is fully reproducible from
`wiki/` on demand, so there's nothing to reconcile by carrying it in git

View file

@ -20,7 +20,8 @@ znajdziesz w [MANUAL.pl.md](MANUAL.pl.md) ([English](MANUAL.md)).
## Struktura katalogów
```
├── libs/ # Zewnętrzne bazy wiedzy tylko do odczytu, kopiowane przez git (w .gitignore)
├── libs/ # Zewnętrzne źródła tylko do odczytu — kopie git (w .gitignore) LUB
│ # konfiguracje konektora (source.yaml) z własnym generowanym indeksem
├── linked/ # Zewnętrzne bazy wiedzy tylko do odczytu, montowane jako dowiązania symboliczne
├── outputs/ # Wygenerowane artefakty, eksporty, skompilowane pliki
├── raw/ # Materiał źródłowy dostarczony przez użytkownika
@ -48,12 +49,14 @@ wygrywa:
wiki/ (najwyższy) ← agent zapisuje tutaj, zawsze wygrywa
linked/A/ (średni) ← zamontowane przez symlink wiki nadrzędne
linked/B/ (niski) ← zamontowane przez symlink wiki nadrzędne
libs/A/ (najniższy) ← kopie zewnętrznych baz wiedzy zarządzane przez git
libs/A/ (najniższy) ← kopie zewnętrznych baz wiedzy zarządzane przez git, albo własny generowany indeks konektora
```
Agent nigdy nie zapisuje do `linked/` ani `libs/`. Aby poprawić treść
nadrzędną, zapisz właściwą wersję w `wiki/` — automatycznie zyskuje
pierwszeństwo.
Agent nigdy nie zapisuje do `linked/` ani do `libs/<name>/` będącego kopią
git. Aby poprawić treść nadrzędną, zapisz właściwą wersję w `wiki/`
automatycznie zyskuje pierwszeństwo. Jedynym wyjątkiem jest `libs/<name>/`
oparty na konektorze (zobacz „Konektory zewnętrznych źródeł i indeksowanie”
poniżej) — agent zarządza jego generowanym indeksem tak samo, jak `wiki/`.
---
@ -139,6 +142,59 @@ własny `log.md`, gdy ma już wystarczająco dużo niezależnej historii;
`wiki/log.md` pozostaje rollupem na poziomie głównym i nigdy nie powtarza
zmiany już zapisanej w dzienniku podkatalogu.
### Konektory zewnętrznych źródeł i indeksowanie (na żądanie)
Folder `libs/<name>/` obsługuje drugi sposób zasilania, obok istniejącej
kopii git: mały, autorski plik `libs/<name>/source.yaml`, który deklaruje
*żywe* zewnętrzne źródło — folder SharePoint, folder Google Drive, zwykły
URL albo inny konektor — którego nie chcesz w pełni kopiować lokalnie:
```yaml
connector: sharepoint
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Wspólny folder raportów zespołu finansowego"
```
Powiedz „index external sources", a agent go przeskanuje, dopasowując
`connector` do dowolnego żywego narzędzia dostępnego w danej sesji
(połączonego narzędzia MCP do Microsoft 365/Google Drive, albo `WebFetch`
dla zwykłego adresu URL), i zbuduje samodzielny, generowany indeks wewnątrz
tego samego `libs/<name>/``index.md`/`entities/`/`graph/`/`log.md`,
odzwierciedlający strukturę `wiki/` opisaną w Rekurencyjnej konwencji
indeksu i dziennika powyżej, ale ograniczony wyłącznie do tego jednego
konektora. To celowa decyzja projektowa: ten indeks **nie** jest wmieszany
w główne `wiki/entities/`/`wiki/graph/edges.json` — pozostaje oddzielony na
warstwie kaskady `libs/`, tak samo jak pliki sklonowanej przez git KB. Sam
`source.yaml` pozostaje wyłącznie twój, agent nigdy go nie zapisuje.
Dwa rozszerzenia na tym fundamencie:
- **Współdzielone, wcześniej zbudowane indeksy.** `source.yaml` może
dodać opcjonalny blok `index:`, który deklaruje, *gdzie już zbudowany
indeks się znajduje* — repozytorium git albo zasób współdzielony, np.
ścieżka sieciowa lub inna lokalizacja dostępna przez konektor:
```yaml
index:
store: git # git | shared
location: "https://github.com/org/finance-index-cache.git"
ref: main # opcjonalnie — branch, tag albo podpowiedź co do podścieżki w tym miejscu
```
Każde uruchomienie najpierw sprawdza tę lokalizację: jeśli indeks już
tam jest, zostaje pobrany — większość osób po prostu czyta to, co już
jest, zamiast budować to samodzielnie. Jeśli go tam jeszcze nie ma, to
normalny stan przy pierwszym uruchomieniu, a nie błąd: kolejne
uruchomienie użytkownika z dostępem do zapisu jest tym, które go tam
tworzy i publikuje — bez osobnego kroku „inicjalizacji".
- **Odczyt vs. zapis, per użytkownik, per źródło.** Czy *ten* użytkownik
może faktycznie przebudować indeks (a nie tylko czytać pobraną/
opublikowaną wersję) to odrębne, lokalne, ignorowane przez git
`libs/<name>/source.local.yaml` — domyślnie tylko do odczytu. Ustawienie
`access: write` w tym pliku włącza dany komputer/użytkownika jako
administratora tego źródła, dzięki czemu zespół może wyznaczyć jedną lub
dwie osoby do utrzymywania źródła, podczas gdy reszta czyta tylko wynik —
bez zbędnego, wielokrotnego przebudowywania i bez potrzeby, żeby każdy
użytkownik miał własną autoryzację konektora.
Zaimplementowane jako Claude Code Skill — zobacz
`.agents/skills/ckb-index-external/SKILL.md`.
### Podwójne linkowanie (Wikilinks + Markdown)
Każde odwołanie krzyżowe używa zarówno `[[Wikilinks]]` (kompatybilnych z
Obsidian), jak i standardowych linków `[markdown](path.md)`. Działa w
@ -326,6 +382,10 @@ słabe punkty przed przejściem dalej. Nigdy nie zapisuje do `wiki/`. Zobacz
ln -s /path/to/other-kb ./linked/my-upstream
git clone https://github.com/org/external-kb ./libs/external-kb
```
Albo, dla żywego źródła, którego nie chcesz w pełni kopiować lokalnie,
umieść zamiast tego `libs/<name>/source.yaml` (zobacz „Konektory
zewnętrznych źródeł i indeksowanie" powyżej) i powiedz „index external
sources".
2. **Wrzuć surowy materiał** do `raw/inbox/` (notatki, linki, artykuły).
@ -357,8 +417,12 @@ synchronizacji.
## Wskazówki
- Nadrzędne bazy wiedzy (`linked/` i `libs/`) **nigdy nie są modyfikowane**
przez agentów.
- Nadrzędne bazy wiedzy (`linked/` i kopie git w `libs/`) **nigdy nie są
modyfikowane** przez agentów. Wyjątkiem jest `libs/<name>/` oparty na
konektorze (ten z `source.yaml`) — agent zarządza jego generowanym
indeksem, ale tylko dla użytkownika, który lokalnie ustawił sobie
`access: write` (zobacz następny punkt); kopia każdego innego
użytkownika pozostaje tylko do odczytu.
- Aby poprawić treść nadrzędną, zapisz poprawną wersję w `wiki/` — ona
wygrywa.
- Używaj `raw/inbox/` dla wszystkiego, co nieprzetworzone; agent czyści ją
@ -367,7 +431,15 @@ synchronizacji.
bieżąco.
- Confidence, quality i freshness pozwalają ufać właściwej treści i
oflagowywać resztę do przeglądu.
- Katalogi `tmp/` i `libs/` są w `.gitignore`. Sam `outputs/` jest śledzony,
- Katalog `tmp/` jest w `.gitignore`, podobnie jak większość `libs/` — ale
nie cały: zawartość kopii git w `libs/<name>/` pozostaje w `.gitignore`
jak dawniej, natomiast `source.yaml` konektora i jego generowany
`index.md`/`entities/`/`graph/`/`log.md` są śledzone, ponieważ to
zsyntetyzowana wiedza warta udostępnienia przez „sync changes", a nie
jednorazowy artefakt budowania. `libs/<name>/source.local.yaml`
(osobiste ustawienie odczytu/zapisu) to jedyny wyjątek, który zostaje w
`.gitignore` razem z nimi — to osobisty stan komputera, nigdy
przeznaczony do synchronizacji. Sam `outputs/` jest śledzony,
ale jego regenerowalne podkatalogi budowania, `outputs/okf/` i
`outputs/starlight/`, są w `.gitignore` — każdy z nich jest w pełni
odtwarzalny z `wiki/` na żądanie, więc nie ma czego uzgadniać, przenosząc

View file

@ -1 +1 @@
1.0.0
1.1.0