From 946619de89d9206e20a3610ca4d2b08814da8fd7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Kope=C4=87?= Date: Mon, 20 Jul 2026 16:20:06 +0200 Subject: [PATCH] Add shared pre-built indexes and per-user read/write access for connector sources source.yaml gains an optional index: block declaring where an already-built index lives (a git repo or a shared resource), so a user can fetch it instead of scanning the live connector from scratch. Whether a given user may actually rebuild/publish an index is now a local, per-user, gitignored source.local.yaml (access: write|read) that defaults to read-only, letting a team designate one or two admins per external source instead of everyone redundantly re-indexing it. ckb-lint's checks against a connector's generated index now respect the same read/write gate. Co-Authored-By: Claude Sonnet 5 --- .agents/skills/ckb-index-external/SKILL.md | 147 ++++++++++++++++++--- .agents/skills/ckb-lint/SKILL.md | 37 ++++-- .gitignore | 3 + AGENTS.md | 11 +- MANUAL.md | 57 +++++++- MANUAL.pl.md | 60 ++++++++- README.md | 34 ++++- README.pl.md | 33 ++++- 8 files changed, 336 insertions(+), 46 deletions(-) diff --git a/.agents/skills/ckb-index-external/SKILL.md b/.agents/skills/ckb-index-external/SKILL.md index 9be6215..ee06cd1 100644 --- a/.agents/skills/ckb-index-external/SKILL.md +++ b/.agents/skills/ckb-index-external/SKILL.md @@ -1,6 +1,6 @@ --- name: ckb-index-external -description: Walk every connector-backed libs// (identified by a source.yaml declaring a live external source — SharePoint, Google Drive, a plain URL, or another connector) and build/refresh a self-contained generated index for it — index.md/entities/graph/log.md, mirroring wiki/'s own shape but scoped entirely to that one connector. Never touches wiki/, never touches source.yaml, never touches a git-copy lib. Use when the user says "index external sources", "index libs", "refresh the external index", or "scan the SharePoint/Drive folder". This is the on-demand workflow from CLAUDE.md/AGENTS.md §4, distinct from `ckb-ingest` (which processes raw/inbox/ into wiki/) and from a plain git-copy libs// (a frozen clone, never touched by any skill). +description: Walk every connector-backed libs// (identified by a source.yaml declaring a live external source — SharePoint, Google Drive, a plain URL, or another connector) and build/refresh a self-contained generated index for it — index.md/entities/graph/log.md, mirroring wiki/'s own shape but scoped entirely to that one connector. The index can optionally be published to (and fetched from) a shared location — a git repo or a shared resource — declared in source.yaml, so most users just read an already-built index instead of re-scanning the live source. Read vs. write access to a given source is a per-user, local-only setting (libs//source.local.yaml, gitignored) — read-only by default, so one or two designated admins can maintain a source for everyone else. Never touches wiki/, never touches source.yaml, never touches a git-copy lib. Use when the user says "index external sources", "index libs", "refresh the external index", or "scan the SharePoint/Drive folder". This is the on-demand workflow from CLAUDE.md/AGENTS.md §4, distinct from `ckb-ingest` (which processes raw/inbox/ into wiki/) and from a plain git-copy libs// (a frozen clone, never touched by any skill). --- # Index external sources skill @@ -23,10 +23,25 @@ KB's own files would. If something in it needs to override or correct what the connector says, that's what writing the corrected version into `wiki/` is for (cascade priority already covers that — `wiki/` always wins). +Two refinements on top of that base design: + +- **Shared, pre-built indexes.** `source.yaml` can declare *where the + already-built index lives* (a git repo, or a shared resource such as a + network path or another connector-reachable location), so a user doesn't + have to build the index from scratch — they just fetch what's already + there. +- **Read vs. write access, per user, per source.** Building/refreshing an + index from the live connector is gated by a **local, per-user** setting + (`libs//source.local.yaml`, never committed) — read-only by + default. This lets a team designate one or two people as the "admin" of + a given external source, while everyone else just consumes whatever + index the admin has already published, without needing their own + connector authorization or triggering redundant/conflicting rebuilds. + This skill only ever writes within a connector-backed `libs//` (never `source.yaml` itself, never a git-copy lib, never anything under `wiki/`) — and it never touches this repo's own git remote; pushing the resulting -changes is a separate, explicit step (see Step 6). +changes is a separate, explicit step (see Step 7). ## Trigger phrases @@ -36,12 +51,53 @@ Use this skill when the user says: - "Refresh the external index" - "Scan the SharePoint/Drive folder" (or similarly names a specific connector) +- "Make me the admin for ``" / "give me write access to ``" + / "I don't want write access to `` anymore" — a lighter-weight + request that only touches that one source's `source.local.yaml` (see + Configuration below) and doesn't necessarily trigger a full re-index in + the same turn unless the user also asks for one. Do **not** use this skill for "Ingest" / "Sync the wiki" (that's `ckb-ingest` — raw material into `wiki/`, unrelated to `libs/`) or for "Lint" (that's `ckb-lint`, which now also health-checks a connector's own generated index, but doesn't build it in the first place). +## Configuration + +### `libs//source.yaml` (shared, tracked in git, same for everyone) + +```yaml +connector: sharepoint # sharepoint | google_drive | web | custom — open string, not a fixed enum +location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports" +description: "Finance team's shared reports folder" # optional + +index: # optional — omit entirely for the original default: the index lives only + # here, in this repo, tracked by this repo's own git (nothing to fetch/publish) + store: git # git | shared — open string, where the built index is published to / fetched from + location: "https://github.com/org/finance-index-cache.git" # a git repo URL (store: git), or a + # filesystem/network path or connector-reachable location (store: shared) + ref: main # optional — branch, tag, or subpath hint within that store +``` + +### `libs//source.local.yaml` (per-user, per-machine, **never committed**) + +```yaml +access: write # write | read — defaults to read if this file is absent or the field is omitted +``` + +This file is intentionally **not** carved out of `.gitignore` — it stays +caught by the existing `libs/*/*` catch-all, exactly like any other +unlisted file inside a connector-backed `libs//`. It's local +machine/user state, the same category as a personal `.env` — never synced, +never reviewed by anyone else, and it's what makes "read vs. write" a +per-user decision rather than a shared one baked into `source.yaml`. + +Unlike `source.yaml`, this skill *may* create or edit `source.local.yaml` +on this user's behalf — but only when they explicitly ask to become (or +stop being) a given source's admin (e.g. "make me the admin for the +finance reports source" / "I don't want write access to X anymore"), never +as a side effect of just running "index external sources." + ## How to run this skill ### Step 1 — Find connector-backed libs @@ -54,14 +110,43 @@ If no `libs//source.yaml` exists anywhere, tell the user there's nothing to index yet and how to add one (a `source.yaml` with `connector` and `location`), and stop — don't fabricate an index from nothing. -### Step 2 — Read the config and resolve the connector +### Step 2 — Read the local access setting + +Read `libs//source.local.yaml` if it exists. `access: write` means +this user is an admin for this one source; anything else (the field is +`read`, or the file doesn't exist at all) means read-only. This check is +**per connector-backed lib** — a user can be write-access for one source +and read-only for another. + +### Step 3 — Fetch a published index, if one is configured + +If `source.yaml` has an `index:` block, fetch/pull the latest published +index from `index.location` (per `index.store`: `git pull`/clone for +`git`, a plain file copy for a filesystem/network `shared` path, or the +matching connector tool for a `shared` location that needs one) and use it +to refresh `libs//{index.md,entities/,graph/,log.md}`. Do this +**regardless of this user's access level** — reading the published index +never requires write access, and it's exactly what lets a read-only user +skip building anything themselves. + +If no `index:` block is configured, there's nothing to fetch — the index +(if any exists yet) already lives directly in `libs//`, same as +before. + +### Step 4 — Decide whether to build or refresh from the live connector + +- **Read-only** (the common case, and the default for anyone who hasn't + set `access: write` locally): stop here. Report what the fetched/existing + index already covers. If nothing has ever been indexed for this source + and this user is read-only, say so plainly — don't scan the live + connector on their behalf. Suggest asking whoever administers this + source to run it, or setting `access: write` locally if this user is + meant to be that admin. +- **Write access**: continue to Step 5 and actually build/refresh the + index from the live connector `location`. + +### Step 5 — Resolve the connector and enumerate/summarize documents -Read `source.yaml`: -```yaml -connector: sharepoint # sharepoint | google_drive | web | custom — open string, not a fixed enum -location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports" -description: "Finance team's shared reports folder" # optional -``` Resolve `connector` to whatever live tool is actually available this session: - `sharepoint` / a Microsoft-hosted location → search (`ToolSearch`) for a @@ -75,8 +160,6 @@ connector needs authorizing (via claude.ai connector settings, or `claude mcp`/`/mcp` for non-claude.ai servers), skip that source, and continue with any others. -### Step 3 — Enumerate and summarize documents - For each document found at the location: - Read enough content to write a real 1-2 sentence `tldr` when the connector allows fetching content. @@ -99,7 +182,7 @@ retention: medium --- ``` -### Step 4 — Update the hub page and graph +### Step 6 — Update the hub page, graph, index, and log Create/update `libs//index.md` — the root routing page for this connector, mirroring `wiki/index.md`'s own role: @@ -119,8 +202,6 @@ edges in `libs//graph/edges.json`, using a `contains`/`part_of`-style edge type — this is fine to be thin or empty for a flat source with no useful structure beyond a document list. -### Step 5 — Update index/log, following the Recursive Index & Log Convention - Update `libs//entities/index.md` and `libs//graph/index.md` (flat bullet lists, no frontmatter, same convention as `wiki/entities/ index.md`/`wiki/graph/index.md`). Log every created/updated page in @@ -128,7 +209,19 @@ index.md`/`wiki/graph/index.md`). Log every created/updated page in (`CLAUDE.md`/`AGENTS.md` §7) — this log is independent of `wiki/log.md`; **nothing under `wiki/` is touched by this skill at all.** -### Step 6 — Remind to review and sync +### Step 7 — Publish, if this source has both write access and a configured index store + +If this user has `access: write` **and** `source.yaml` has an `index:` +block, push the refreshed `libs//{index.md,entities/,graph/,log.md}` +back out to `index.location` (`git push` for `store: git`, a file copy +back for a filesystem/network `store: shared` path, or the matching +connector tool for a `shared` location that needs one) — so every other +user's next fetch (Step 3) picks up the update. If no `index:` block is +configured, there's nothing to publish; the refreshed files staying inside +this repo's own `libs//` (tracked by this repo's own git) *is* the +publish step, same as the original default behavior. + +### Step 8 — Remind to review and sync This is always the last step, every time this skill makes any change at all. Close with a short reminder — do not sync or push anything yourself @@ -139,17 +232,30 @@ here: > once you're happy with them, say 'sync changes' to commit and push them > to `origin`." -If nothing was actually indexed (no connector-backed libs found, or every -source was skipped for lack of authorization), skip this reminder. +If nothing was actually indexed or fetched (no connector-backed libs +found, every source was skipped for lack of authorization, or every source +was read-only with nothing new to fetch), skip this reminder. ## Edge cases - **No connector-backed `libs//` exists** — report that and explain how to add one; don't touch anything. +- **`source.local.yaml` doesn't exist yet for a source** — treat as + read-only; this is the correct default, not a misconfiguration worth + flagging loudly. Mention once, briefly, how to become the admin + (`access: write` in that file) if the user seems to expect a rebuild. +- **Everyone is read-only and nothing has ever been published** — report + that clearly rather than silently doing nothing; this is the one case + worth flagging as a real gap (nobody can currently index this source). - **Connector not authorized this session** — report which one and how to authorize it (claude.ai connector settings / `claude mcp` / `/mcp`), skip that source, keep going with the rest. Never ask the user for tokens or callback URLs. +- **Fetching a published index (Step 3) fails or the remote is + unreachable** — report it and fall back to whatever's already cached + locally in `libs//` from a previous fetch, rather than blocking + the whole run; don't let a transient network issue block a read-only + user from seeing the last-known index. - **Re-running against an already-indexed source** — refresh existing entity pages in place (update `tldr`/`last_updated`/`confidence`); never duplicate a document's page. @@ -162,9 +268,14 @@ source was skipped for lack of authorization), skip this reminder. one run and explicitly report what was skipped (e.g. "42 of ~300 documents summarized this pass — re-run to continue"). Never silently truncate while implying full coverage. +- **Write access but publishing (Step 7) conflicts with a newer version + someone else already pushed** — this is why Step 3 (fetch) always runs + first, even for write-access users: rebuild on top of the latest fetched + state rather than blindly overwriting it. If a real conflict still shows + up at publish time, report it rather than force-pushing over it. - **`libs//` has both real content files and a `source.yaml`** — ambiguous and out of scope for this skill to resolve; report it and stop - for that folder (this is the same case `ckb-lint`'s new external-source + for that folder (this is the same case `ckb-lint`'s external-source config check flags). - **`source.yaml` is missing `connector` or `location`** — report it as malformed and skip that source; don't guess at intent. diff --git a/.agents/skills/ckb-lint/SKILL.md b/.agents/skills/ckb-lint/SKILL.md index 654dbd1..1dd6dde 100644 --- a/.agents/skills/ckb-lint/SKILL.md +++ b/.agents/skills/ckb-lint/SKILL.md @@ -19,12 +19,16 @@ actually invoked — on demand, or on a schedule if the user has set one up. This skill only ever writes within `wiki/` (including moves into `wiki/archived/`) and, for any connector-backed `libs//` (one with a -`source.yaml` — see `CLAUDE.md`/`AGENTS.md` §1/§4), within that connector's +`source.yaml` — see `CLAUDE.md`/`AGENTS.md` §1/§4) where this user has +local `access: write` (see `ckb-index-external`), within that connector's own agent-owned generated index (`index.md`/`entities/`/`graph/`/`log.md`). -It never touches `linked/`, a git-copy `libs//`, or a connector's -`source.yaml` itself — those stay immutable upstream sources of truth — and -it never touches this repo's own git remote; pushing the resulting changes -is a separate, explicit step (see the final step below). +For a connector-backed `libs//` where this user is read-only (the +default), findings against that index are reported only, never auto-fixed +— the same access gate `ckb-index-external` applies to rebuilding applies +here too. It never touches `linked/`, a git-copy `libs//`, or a +connector's `source.yaml`/`source.local.yaml` — those stay immutable or +user-owned — and it never touches this repo's own git remote; pushing the +resulting changes is a separate, explicit step (see the final step below). ## Trigger phrases @@ -46,10 +50,12 @@ check's ability to reason about it, so conformance comes first. Checks 1-9 below are written against `wiki/`, but the same checks also run a second time against each connector-backed `libs//`'s own generated index — scoped entirely to that one connector's `index.md`/`entities/`/ -`graph/`/`log.md`. Findings and fixes there are logged in that connector's -own `log.md`, never in `wiki/log.md`. A git-copy `libs//` (no -`source.yaml`) and any `source.yaml` itself are never touched by any of -these checks. +`graph/`/`log.md`. Auto-fixes there only happen where this user has local +`access: write` for that source (see `ckb-index-external`); everywhere +read-only, findings are reported but not applied. Findings and fixes there +are logged in that connector's own `log.md`, never in `wiki/log.md`. A +git-copy `libs//` (no `source.yaml`) and any `source.yaml`/ +`source.local.yaml` are never touched by any of these checks. ### 1 — Conformance check @@ -116,10 +122,15 @@ doesn't need an Error Book entry — this is for patterns, not incidents. ### 10 — External source config check For each `libs//source.yaml`, verify it has a non-empty `connector` -and `location` — report only, this one file is never edited by any skill. -Also flag (report only) a `libs//` that ambiguously has both real -content files and a `source.yaml` — that's a configuration conflict for the -user to resolve, not something to guess at. +and `location` — report only, this file is never edited by any skill. If +an `index:` block is present, verify it has a non-empty `store` and +`location` too. Also flag (report only) a `libs//` that ambiguously +has both real content files and a `source.yaml` — that's a configuration +conflict for the user to resolve, not something to guess at. + +Don't flag a missing `libs//source.local.yaml` as an issue — its +absence is the correct, read-only default (see `ckb-index-external`), not +a gap to report or fix. ### Auto-fix vs. report diff --git a/.gitignore b/.gitignore index db01759..40592e5 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,9 @@ libs/*/* !libs/*/log.md !libs/*/entities/ !libs/*/graph/ +# Per-user local override (e.g. source.local.yaml's access: write) — never shared, stays ignored +# by the libs/*/* catch-all above; listed explicitly for clarity, not because it changes behavior. +libs/*/*.local.yaml tmp/* !tmp/.gitadd outputs/starlight diff --git a/AGENTS.md b/AGENTS.md index 2929283..a76c355 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,11 +17,14 @@ The root directory contains exactly seven top-level entries. You must maintain t ``` ├── libs/ # Read-only external sources, one of two kinds per / subfolder: │ └── / # - GIT-COPY: a git-managed clone/ZIP unpack, gitignored, fully immutable — never write here. -│ # - CONNECTOR: identified by a user-authored source.yaml (connector + location). +│ # - CONNECTOR: identified by a user-authored source.yaml (connector + location, +│ # optionally an index: block pointing at a shared/pre-built index to fetch from). │ # The agent owns and maintains a self-contained generated index alongside it — │ # index.md/entities/graph/log.md, mirroring wiki/'s own shape but scoped entirely │ # to this one connector. See §4 EXTERNAL SOURCE INDEXING. source.yaml itself stays -│ # user-only, same as everything in a git-copy lib. +│ # user-only, same as everything in a git-copy lib. Whether *this* user may rebuild +│ # it (vs. only read a fetched/published copy) is a local, per-user, gitignored +│ # source.local.yaml — read-only by default. ├── linked/ # SYMLINKS ONLY. Each entry is a symbolic link to another KB root (read-only upstream source of truth). │ └── / # Individual upstream knowledge base (immutable — never write here). ├── outputs/ # MANAGED BY AGENT. Generated artifacts, exports, compiled files produced from the wiki. @@ -127,7 +130,7 @@ When the user says "Ingest", "Sync the wiki", or "Update the Wiki" (for syncing ## 4. EXTERNAL SOURCE INDEXING (TRIGGERED ON DEMAND) -When the user says "Index external sources" (or "index libs", "refresh the external index"), run the **ckb-index-external** Claude Code Skill — see `.agents/skills/ckb-index-external/SKILL.md` — rather than following inline steps here, so the full procedure only loads into context when actually invoked. It walks every connector-backed `libs//` (one with a `source.yaml` — see §1), resolves the declared connector to whatever live tool is available this session (SharePoint/Drive MCP connector, or `WebFetch` for a plain URL), and builds/refreshes that connector's own self-contained `index.md`/`entities/`/`graph/`/`log.md`. This never touches `wiki/`, never touches `source.yaml`, and never touches a git-copy lib. +When the user says "Index external sources" (or "index libs", "refresh the external index"), run the **ckb-index-external** Claude Code Skill — see `.agents/skills/ckb-index-external/SKILL.md` — rather than following inline steps here, so the full procedure only loads into context when actually invoked. It walks every connector-backed `libs//` (one with a `source.yaml` — see §1), fetches a shared/pre-built index if `source.yaml` declares one (`index.store`/`index.location` — git or a shared resource), and — only if this user has local `access: write` in `libs//source.local.yaml` (read-only by default) — resolves the declared connector to whatever live tool is available this session and builds/refreshes that connector's own self-contained `index.md`/`entities/`/`graph/`/`log.md`, publishing it back to the shared store if one is configured. This never touches `wiki/`, never touches `source.yaml`, and never touches a git-copy lib. --- @@ -156,6 +159,8 @@ You must **never** write, modify, move, or delete any file or directory inside ` **Exception — connector-backed `libs//`:** identified by the presence of a `source.yaml` (see §1). Its `source.yaml` is user-authored and stays just as untouchable as anything else here. But everything else in that folder — `index.md`, `entities/`, `graph/`, `log.md` — is a generated index the agent owns and maintains exactly as it would `wiki/`, built and refreshed by the `ckb-index-external` skill (§4). This exception applies only to a `libs//` that has a `source.yaml`; a plain git-copy lib has no such carve-out. +Within that exception, two things the agent may always do regardless of this user's access level: fetch a shared/pre-built index down into `libs//` if `source.yaml` declares one, and read whatever's cached there. Actually rebuilding it from the live connector — and publishing that rebuild back to a shared store — is gated by a separate, local, per-user `libs//source.local.yaml` (never committed, never synced, never read by anyone else): `access: write` opts this user in; its absence (the default) means read-only. Unlike `source.yaml`, the agent *may* create or edit `source.local.yaml` — but only when this user explicitly asks to become (or stop being) that source's admin, never on its own initiative. + ### Rule B: The Wiki Change Log (`wiki/log.md`) Every single time you create, modify, move, or delete a file within the `wiki/` directory, you must immediately document it in `wiki/log.md` before proceeding. - **Ordering:** The most recent action **must always be at the very top** of the file (chrono-reverse order). diff --git a/MANUAL.md b/MANUAL.md index d149f07..31e366d 100644 --- a/MANUAL.md +++ b/MANUAL.md @@ -88,14 +88,29 @@ which stay entirely read-only. There are three ways to mount one: See [§6](#6-use-cases) for a worked example and what the result looks like. + Two things worth knowing up front about a connector source: + - **You don't have to build the index yourself.** `source.yaml` can add + an `index:` block pointing at an already-built index — a git repo, or + a shared resource — so you just fetch what someone else already + indexed instead of scanning the live source yourself. + - **Building/refreshing is opt-in per person, per source.** By default, + everyone is read-only for a connector source — nobody's agent will + scan the live connector on their behalf unless they've explicitly said + so. Say "make me the admin for ``" to opt yourself in (this + writes a local, personal `libs//source.local.yaml` — never + committed, never seen by anyone else). This is deliberate: it lets one + or two people maintain a source for the whole team instead of everyone + redundantly re-scanning it. + Either way, once it's mounted, just ask questions normally — the agent checks your local `wiki/` first, then walks `linked/`, then `libs/`, and uses whichever has the answer. You never edit files inside `linked/` or a git-copy `libs//` directly; if something there is wrong or outdated, you fix it by writing the corrected version into your own local `wiki/`, which always wins. (A connector-backed `libs//` is the one place the -agent *does* write on your behalf — see [§6](#6-use-cases) — but even -there, `source.yaml` itself stays yours to edit, never the agent's.) +agent *does* write on your behalf — see [§6](#6-use-cases) — but only its +generated index, and only the build/refresh part if you're that source's +admin; `source.yaml` itself always stays yours to edit, never the agent's.) --- @@ -455,6 +470,39 @@ disappeared from the source (it flags it instead, so a later "Lint" pass archives it naturally). Implemented by the `ckb-index-external` skill — `.agents/skills/ckb-index-external/SKILL.md`. +**Who's allowed to build it, and where it's shared.** By default, nobody +has write access to a connector source until they say so — this keeps a +team of, say, ten people from all redundantly scanning the same SharePoint +folder. Say: + +> "Make me the admin for finance-reports." + +This writes a personal `libs/finance-reports/source.local.yaml` with +`access: write` — never committed, never seen by your teammates. Anyone +without that file is read-only for that source: if they say "index +external sources," the agent won't touch the live connector on their +behalf at all — it just reports what's already indexed (or tells them +plainly that nothing has been indexed yet and who to ask). + +If the finance team wants everyone reading the *same* index rather than +each maintaining their own local copy of it inside their own KB, the admin +adds an `index:` block to the shared `source.yaml`: + +```yaml +index: + store: git + location: "https://github.com/finance-team/index-cache.git" +``` + +Now, whenever *anyone* says "index external sources," the agent first +fetches whatever's already published there — read-only users stop right +there; the admin also rebuilds from the live connector and pushes the +refreshed index back to that same location, so the next person's fetch +picks it up. Leave the `index:` block out entirely (the simplest setup, +and the right default for a single small team) and the index just lives +directly inside `libs/finance-reports/` in this KB's own repo, shared the +normal way via "sync changes" — exactly like the plain example above. + --- ## 7. What's agent-generated vs. what you can edit @@ -473,8 +521,9 @@ graph stay in sync with what you changed. | `raw/archive//` | Agent | Auto-filed copy of what you dropped in `raw/inbox/`, organised by ingestion date. Don't hand-file here — let Ingest do it, so the date and pairing with the log entry stay accurate. | | `linked//` | **You** (you create the symlink) | Points at another KB's real files, which live and get edited *in that other repo* — never here. The agent must never write inside `linked/`. | | `libs//` (git-copy, no `source.yaml`) | **You** (you `git clone`) | A frozen copy of an external KB. Update it by re-pulling that repo yourself, not by hand-editing files here. The agent must never write inside it. | -| `libs//source.yaml` (connector) | **You, only** | Declares the connector and location. The agent reads it but never writes it — same as anything else upstream. | -| `libs//{index.md,entities/,graph/,log.md}` (connector) | Agent-generated, **you can freely edit** | The agent's own index of that one connector's source, built by "Index external sources." Structurally the same deal as the main `wiki/` row below — feel free to correct an entry by hand, then run "Lint" (it now also checks connector-backed indexes). Scoped entirely to that connector; never blended into `wiki/`. | +| `libs//source.yaml` (connector) | **You, only** | Declares the connector, location, and optionally where a shared/pre-built index lives (`index:`). The agent reads it but never writes it — same as anything else upstream. | +| `libs//source.local.yaml` (connector) | **You** (or the agent, only when you explicitly ask to become/stop being that source's admin) | Per-person, per-machine `access: write`/`read` setting — never committed, never seen by anyone else. Absent = read-only, the default. | +| `libs//{index.md,entities/,graph/,log.md}` (connector) | Agent-generated, **you can freely edit** | The agent's own index of that one connector's source, built/refreshed by "Index external sources" — but only if you have `access: write` locally; read-only users just get a fetched copy. Structurally the same deal as the main `wiki/` row below — feel free to correct an entry by hand, then run "Lint" (it now also checks connector-backed indexes, respecting the same read/write split). Scoped entirely to that connector; never blended into `wiki/`. | | `wiki/` (pages, `index.md`, `overview.md`, `log.md`, `error-book.md`, `entities/`, `graph/`) | Agent-generated, **you can freely edit** | This is the one place the agent both writes and expects you might too. Feel free to correct a page by hand — just keep the frontmatter fields intact (or update `last_updated`), and run Lint afterward if you touched something the index/graph/log reference. | | `outputs/okf/`, `outputs/starlight/` | Agent, **fully regenerated** | Don't hand-edit — these are gitignored build artifacts, silently overwritten the next time you export. If something's wrong, fix the wiki page it came from and re-export. | | `outputs/teaching//` | Agent, semi-persistent state | `plan.md`/`progress.md` the teaching skill reads and writes across sessions. You can look at them any time; hand-editing is possible but may confuse "what's next" tracking — safer to tell the agent what you want changed and let it update the files. | diff --git a/MANUAL.pl.md b/MANUAL.pl.md index df06ea1..837d376 100644 --- a/MANUAL.pl.md +++ b/MANUAL.pl.md @@ -91,6 +91,21 @@ podpięcia: samego folderu `libs//`. Zobacz [§6](#6-przykłady-użycia) po omówiony przykład i to, jak wygląda wynik. + Dwie rzeczy warto wiedzieć z góry o źródle typu konektor: + - **Nie musisz sam budować indeksu.** `source.yaml` może dodać blok + `index:` wskazujący na już zbudowany indeks — repozytorium git albo + zasób współdzielony — dzięki czemu po prostu pobierasz to, co ktoś + inny już zaindeksował, zamiast samodzielnie skanować żywe źródło. + - **Budowanie/odświeżanie jest opcjonalne, per osoba, per źródło.** + Domyślnie każdy jest tylko-do-odczytu dla źródła typu konektor — + agent nikogo nie przeskanuje żywego konektora w jego imieniu, jeśli + wyraźnie tego nie zadeklarował. Powiedz „make me the admin for + ``", żeby się na to zapisać (tworzy to lokalny, osobisty plik + `libs//source.local.yaml` — nigdy niecommitowany, nigdy + niewidoczny dla współpracowników). To celowe: pozwala jednej lub dwóm + osobom utrzymywać źródło dla całego zespołu, zamiast żeby każdy + redundantnie je skanował. + Niezależnie od sposobu, po podpięciu wystarczy normalnie zadawać pytania — agent sprawdza najpierw twoją lokalną `wiki/`, potem przechodzi przez `linked/`, potem `libs/`, i korzysta z tego, co ma odpowiedź. Nigdy nie @@ -99,7 +114,9 @@ bezpośrednio; jeśli coś tam jest błędne lub nieaktualne, poprawiasz to, zapisując poprawioną wersję we własnej lokalnej `wiki/`, która zawsze wygrywa. (`libs//` oparty na konektorze to jedyne miejsce, gdzie agent *sam* zapisuje w twoim imieniu — zobacz [§6](#6-przykłady-użycia) — -ale nawet tam `source.yaml` pozostaje twój do edycji, nigdy agenta.) +ale tylko swój generowany indeks, i tylko część budowania/odświeżania, +jeśli jesteś administratorem tego źródła; `source.yaml` zawsze pozostaje +twój do edycji, nigdy agenta.) --- @@ -476,6 +493,42 @@ kolejny przebieg „Lint” zarchiwizował ją naturalnie). Zaimplementowane przez skill `ckb-index-external` — `.agents/skills/ckb-index-external/SKILL.md`. +**Kto może go budować i gdzie jest współdzielony.** Domyślnie nikt nie ma +dostępu do zapisu w źródle typu konektor, dopóki tego nie zadeklaruje — to +chroni zespół dziesięciu osób przed redundantnym skanowaniem tego samego +folderu SharePoint. Powiedz: + +> „Make me the admin for finance-reports.” + +To zapisuje osobisty `libs/finance-reports/source.local.yaml` z +`access: write` — nigdy niecommitowany, nigdy niewidoczny dla +współpracowników. Każdy bez tego pliku jest tylko-do-odczytu dla tego +źródła: jeśli powie „index external sources”, agent w jego imieniu w +ogóle nie dotknie żywego konektora — po prostu zgłosi, co już +zaindeksowano (albo powie wprost, że nic jeszcze nie zaindeksowano i kogo +o to zapytać). + +Jeśli zespół finansowy chce, żeby wszyscy czytali *ten sam* indeks, a nie +każdy utrzymywał własną lokalną kopię w swojej własnej KB, administrator +dodaje blok `index:` do współdzielonego `source.yaml`: + +```yaml +index: + store: git + location: "https://github.com/finance-team/index-cache.git" +``` + +Teraz, kiedy *ktokolwiek* powie „index external sources”, agent najpierw +pobiera to, co już zostało opublikowane — użytkownicy tylko-do-odczytu +zatrzymują się w tym miejscu; administrator dodatkowo przebudowuje indeks +z żywego konektora i wypycha odświeżoną wersję do tej samej lokalizacji, +żeby kolejne pobranie innej osoby ją uwzględniło. Pomiń blok `index:` w +ogóle (najprostsza konfiguracja, właściwy domyślny wybór dla jednego +małego zespołu), a indeks po prostu żyje bezpośrednio wewnątrz +`libs/finance-reports/` we własnym repozytorium tej KB, współdzielony w +normalny sposób przez „sync changes” — zupełnie jak w prostym przykładzie +powyżej. + --- ## 7. Co jest generowane przez agenta, a co możesz edytować @@ -495,8 +548,9 @@ zmieniłeś. | `raw/archive//` | Agent | Automatycznie zarchiwizowana kopia tego, co wrzuciłeś do `raw/inbox/`, uporządkowana według daty ingestu. Nie umieszczaj tu plików ręcznie — pozwól, żeby zrobił to Ingest, tak by data i powiązanie z wpisem w dzienniku były poprawne. | | `linked//` | **Ty** (tworzysz dowiązanie symboliczne) | Wskazuje na rzeczywiste pliki innej KB, które żyją i są edytowane *w tamtym repozytorium* — nigdy tutaj. Agent nigdy nie może zapisywać wewnątrz `linked/`. | | `libs//` (kopia git, bez `source.yaml`) | **Ty** (robisz `git clone`) | Zamrożona kopia zewnętrznej KB. Aktualizujesz ją, ponownie pobierając to repozytorium samodzielnie, a nie ręcznie edytując pliki tutaj. Agent nigdy nie może zapisywać wewnątrz niej. | -| `libs//source.yaml` (konektor) | **Tylko ty** | Deklaruje konektor i lokalizację. Agent go czyta, ale nigdy nie zapisuje — tak jak wszystko inne nadrzędne. | -| `libs//{index.md,entities/,graph/,log.md}` (konektor) | Generowane przez agenta, **możesz swobodnie edytować** | Własny indeks agenta dla tego jednego źródła konektora, budowany przez „Index external sources”. Strukturalnie ta sama zasada jak przy wierszu `wiki/` poniżej — śmiało popraw wpis ręcznie, a potem uruchom „Lint” (teraz sprawdza też indeksy oparte na konektorach). Ograniczone wyłącznie do tego konektora; nigdy nie wmieszane w `wiki/`. | +| `libs//source.yaml` (konektor) | **Tylko ty** | Deklaruje konektor, lokalizację i opcjonalnie gdzie znajduje się współdzielony/wcześniej zbudowany indeks (`index:`). Agent go czyta, ale nigdy nie zapisuje — tak jak wszystko inne nadrzędne. | +| `libs//source.local.yaml` (konektor) | **Ty** (albo agent, tylko gdy wyraźnie poprosisz o zostanie/przestanie bycia administratorem tego źródła) | Osobiste, per-komputer ustawienie `access: write`/`read` — nigdy niecommitowane, nigdy niewidoczne dla innych. Brak = tylko do odczytu, domyślnie. | +| `libs//{index.md,entities/,graph/,log.md}` (konektor) | Generowane przez agenta, **możesz swobodnie edytować** | Własny indeks agenta dla tego jednego źródła konektora, budowany/odświeżany przez „Index external sources” — ale tylko jeśli masz lokalnie `access: write`; użytkownicy tylko-do-odczytu dostają po prostu pobraną kopię. Strukturalnie ta sama zasada jak przy wierszu `wiki/` poniżej — śmiało popraw wpis ręcznie, a potem uruchom „Lint” (teraz sprawdza też indeksy oparte na konektorach, respektując ten sam podział odczyt/zapis). Ograniczone wyłącznie do tego konektora; nigdy nie wmieszane w `wiki/`. | | `wiki/` (strony, `index.md`, `overview.md`, `log.md`, `error-book.md`, `entities/`, `graph/`) | Generowane przez agenta, **możesz swobodnie edytować** | To jedyne miejsce, w którym zarówno agent zapisuje, jak i spodziewa się, że ty też możesz. Śmiało popraw stronę ręcznie — zachowaj tylko pola frontmatteru (lub zaktualizuj `last_updated`) i uruchom potem Lint, jeśli dotknąłeś czegoś, do czego odwołuje się indeks/graf/dziennik. | | `outputs/okf/`, `outputs/starlight/` | Agent, **w pełni regenerowane** | Nie edytuj ręcznie — to zignorowane przez git artefakty budowania, cicho nadpisywane przy każdym kolejnym eksporcie. Jeśli coś jest nie tak, popraw stronę wiki, z której to pochodzi, i wyeksportuj ponownie. | | `outputs/teaching//` | Agent, stan półtrwały | `plan.md`/`progress.md`, które skill do nauczania czyta i zapisuje między sesjami. Możesz je oglądać kiedy chcesz; ręczna edycja jest możliwa, ale może pomieszać śledzenie „co dalej” — bezpieczniej powiedzieć agentowi, co chcesz zmienić, i pozwolić mu zaktualizować pliki. | diff --git a/README.md b/README.md index 7481747..e7026c4 100644 --- a/README.md +++ b/README.md @@ -149,7 +149,30 @@ connector. This is a deliberate design choice: the index is **not** blended into the main `wiki/entities/`/`wiki/graph/edges.json` — it stays separated at the `libs/` cascade layer, the same way a git-cloned KB's own files already are. `source.yaml` itself stays user-only, never written by the -agent. Implemented as a Claude Code Skill — see +agent. + +Two refinements on top of that: + +- **Shared, pre-built indexes.** `source.yaml` can add an optional `index:` + block declaring *where the already-built index lives* — a git repo, or a + shared resource such as a network path or another connector-reachable + location — so most people just fetch what's already there instead of + building it themselves: + ```yaml + index: + store: git # git | shared + location: "https://github.com/org/finance-index-cache.git" + ``` +- **Read vs. write, per user, per source.** Whether *this* user can + actually rebuild an index (versus only read a fetched/published one) is a + separate, local, gitignored `libs//source.local.yaml` — read-only + by default. Setting `access: write` there opts a given machine/user in as + that source's admin, so a team can designate one or two people to + maintain a source while everyone else just reads the result — no + redundant rebuilding, no need for every user to have their own connector + authorization. + +Implemented as a Claude Code Skill — see `.agents/skills/ckb-index-external/SKILL.md`. ### Dual-Linking (Wikilinks + Markdown) @@ -346,7 +369,9 @@ copy to keep in sync. - Upstream KBs (`linked/` and git-copy `libs/`) are **never modified** by agents. A connector-backed `libs//` (one with a `source.yaml`) is - the one exception — the agent owns and maintains its generated index. + the one exception — the agent owns and maintains its generated index, + but only for a user who's opted themselves into `access: write` locally + (see the next point); everyone else's copy stays read-only. - To correct upstream content, write the correct version in `wiki/` — it wins. - Use `raw/inbox/` for anything unprocessed; the agent clears it on ingest. - The `wiki/index.md` routing table is the most important file — keep it current. @@ -357,7 +382,10 @@ copy to keep in sync. before, while a connector-backed `libs//`'s `source.yaml` and its generated `index.md`/`entities/`/`graph/`/`log.md` are tracked, since they're synthesized knowledge worth sharing via "sync changes," not a - disposable build artifact. `outputs/` itself is + disposable build artifact. `libs//source.local.yaml` (per-user + read/write setting) is the one exception that stays gitignored right + alongside them — it's personal machine state, never meant to sync. + `outputs/` itself is tracked, but its regenerated build subdirectories, `outputs/okf/` and `outputs/starlight/`, are gitignored — each is fully reproducible from `wiki/` on demand, so there's nothing to reconcile by carrying it in git diff --git a/README.pl.md b/README.pl.md index 9e9265b..3e3d7e5 100644 --- a/README.pl.md +++ b/README.pl.md @@ -163,6 +163,30 @@ konektora. To celowa decyzja projektowa: ten indeks **nie** jest wmieszany w główne `wiki/entities/`/`wiki/graph/edges.json` — pozostaje oddzielony na warstwie kaskady `libs/`, tak samo jak pliki sklonowanej przez git KB. Sam `source.yaml` pozostaje wyłącznie twój, agent nigdy go nie zapisuje. + +Dwa rozszerzenia na tym fundamencie: + +- **Współdzielone, wcześniej zbudowane indeksy.** `source.yaml` może + dodać opcjonalny blok `index:`, który deklaruje, *gdzie już zbudowany + indeks się znajduje* — repozytorium git albo zasób współdzielony, np. + ścieżka sieciowa lub inna lokalizacja dostępna przez konektor — dzięki + czemu większość osób po prostu pobiera to, co już jest, zamiast budować + to samodzielnie: + ```yaml + index: + store: git # git | shared + location: "https://github.com/org/finance-index-cache.git" + ``` +- **Odczyt vs. zapis, per użytkownik, per źródło.** Czy *ten* użytkownik + może faktycznie przebudować indeks (a nie tylko czytać pobraną/ + opublikowaną wersję) to odrębne, lokalne, ignorowane przez git + `libs//source.local.yaml` — domyślnie tylko do odczytu. Ustawienie + `access: write` w tym pliku włącza dany komputer/użytkownika jako + administratora tego źródła, dzięki czemu zespół może wyznaczyć jedną lub + dwie osoby do utrzymywania źródła, podczas gdy reszta czyta tylko wynik — + bez zbędnego, wielokrotnego przebudowywania i bez potrzeby, żeby każdy + użytkownik miał własną autoryzację konektora. + Zaimplementowane jako Claude Code Skill — zobacz `.agents/skills/ckb-index-external/SKILL.md`. @@ -391,7 +415,9 @@ synchronizacji. - Nadrzędne bazy wiedzy (`linked/` i kopie git w `libs/`) **nigdy nie są modyfikowane** przez agentów. Wyjątkiem jest `libs//` oparty na konektorze (ten z `source.yaml`) — agent zarządza jego generowanym - indeksem. + indeksem, ale tylko dla użytkownika, który lokalnie ustawił sobie + `access: write` (zobacz następny punkt); kopia każdego innego + użytkownika pozostaje tylko do odczytu. - Aby poprawić treść nadrzędną, zapisz poprawną wersję w `wiki/` — ona wygrywa. - Używaj `raw/inbox/` dla wszystkiego, co nieprzetworzone; agent czyści ją @@ -405,7 +431,10 @@ synchronizacji. jak dawniej, natomiast `source.yaml` konektora i jego generowany `index.md`/`entities/`/`graph/`/`log.md` są śledzone, ponieważ to zsyntetyzowana wiedza warta udostępnienia przez „sync changes", a nie - jednorazowy artefakt budowania. Sam `outputs/` jest śledzony, + jednorazowy artefakt budowania. `libs//source.local.yaml` + (osobiste ustawienie odczytu/zapisu) to jedyny wyjątek, który zostaje w + `.gitignore` razem z nimi — to osobisty stan komputera, nigdy + przeznaczony do synchronizacji. Sam `outputs/` jest śledzony, ale jego regenerowalne podkatalogi budowania, `outputs/okf/` i `outputs/starlight/`, są w `.gitignore` — każdy z nich jest w pełni odtwarzalny z `wiki/` na żądanie, więc nie ma czego uzgadniać, przenosząc