ckb/.agents/skills/ckb-index-external/SKILL.md

name

description

ckb-index-external

Walk every connector-backed libs/<name>/ (identified by a source.yaml declaring a live external source — SharePoint, Google Drive, a plain URL, or another connector) and build/refresh a self-contained generated index for it — index.md/entities/graph/log.md, mirroring wiki/'s own shape but scoped entirely to that one connector. The index can optionally be published to (and fetched from) a shared location — a git repo or a shared resource — declared in source.yaml, so most users just read an already-built index instead of re-scanning the live source. Read vs. write access to a given source is a per-user, local-only setting (libs/<name>/source.local.yaml, gitignored) — read-only by default, so one or two designated admins can maintain a source for everyone else. Never touches wiki/, never touches source.yaml, never touches a git-copy lib. Use when the user says "index external sources", "index libs", "refresh the external index", or "scan the SharePoint/Drive folder". This is the on-demand workflow from CLAUDE.md/AGENTS.md §4, distinct from `ckb-ingest` (which processes raw/inbox/ into wiki/) and from a plain git-copy libs/<name>/ (a frozen clone, never touched by any skill).

Index external sources skill

Purpose

Give libs/<name>/ a second population mode alongside the existing git-copy one: a libs/<name>/source.yaml declares a live external source — a SharePoint folder, a Google Drive folder, a plain URL, or another connector — that the user doesn't want to fully mirror locally. This skill walks that source and builds a small, self-contained index of what's there: one entity page per document found, plus the same index.md/graph/log.md shape wiki/ already uses, rooted at libs/<name>/ instead.

This is deliberately not blended into the main wiki/entities/ or wiki/graph/edges.json — the generated index lives entirely inside its own libs/<name>/, at the lowest cascade layer, the same way a git-cloned KB's own files would. If something in it needs to override or correct what the connector says, that's what writing the corrected version into wiki/ is for (cascade priority already covers that — wiki/ always wins).

Two refinements on top of that base design:

• Shared, pre-built indexes. source.yaml can declare where the already-built index lives (a git repo, or a shared resource such as a network path or another connector-reachable location). Every run checks that location: if it already has an index, fetch it; if it doesn't yet, a write-access user's run is what creates it there for the first time. Either way, a read-only user just gets whatever's already there — they never have to build the index from scratch themselves.
• Read vs. write access, per user, per source. Building/refreshing an index from the live connector is gated by a local, per-user setting (libs/<name>/source.local.yaml, never committed) — read-only by default. This lets a team designate one or two people as the "admin" of a given external source, while everyone else just consumes whatever index the admin has already published, without needing their own connector authorization or triggering redundant/conflicting rebuilds.

This skill only ever writes within a connector-backed libs/<name>/ (never source.yaml itself, never a git-copy lib, never anything under wiki/) — and it never touches this repo's own git remote; pushing the resulting changes is a separate, explicit step (see Step 7).

Trigger phrases

Use this skill when the user says:

• "Index external sources"
• "Index libs"
• "Refresh the external index"
• "Scan the SharePoint/Drive folder" (or similarly names a specific connector)
• "Make me the admin for <source>" / "give me write access to <source>" / "I don't want write access to <source> anymore" — a lighter-weight request that only touches that one source's source.local.yaml (see Configuration below) and doesn't necessarily trigger a full re-index in the same turn unless the user also asks for one.

Do not use this skill for "Ingest" / "Sync the wiki" (that's ckb-ingest — raw material into wiki/, unrelated to libs/) or for "Lint" (that's ckb-lint, which now also health-checks a connector's own generated index, but doesn't build it in the first place).

Configuration

libs/<name>/source.yaml (shared, tracked in git, same for everyone)

connector: sharepoint       # sharepoint | google_drive | web | custom — open string, not a fixed enum
location: "https://contoso.sharepoint.com/sites/Finance/Shared Documents/Reports"
description: "Finance team's shared reports folder"   # optional

index:                      # optional — omit entirely for the original default: the index lives only
                             #   here, in this repo, tracked by this repo's own git (nothing to fetch/publish)
  store: git                # git | shared — open string, where the built index is published to / fetched from
  location: "https://github.com/org/finance-index-cache.git"   # a git repo URL (store: git), or a
                             #   filesystem/network path or connector-reachable location (store: shared)
  ref: main                 # optional — branch, tag, or subpath hint within that store

libs/<name>/source.local.yaml (per-user, per-machine, never committed)

access: write                # write | read — defaults to read if this file is absent or the field is omitted

This file is intentionally not carved out of .gitignore — it stays caught by the existing libs/*/* catch-all, exactly like any other unlisted file inside a connector-backed libs/<name>/. It's local machine/user state, the same category as a personal .env — never synced, never reviewed by anyone else, and it's what makes "read vs. write" a per-user decision rather than a shared one baked into source.yaml.

Unlike source.yaml, this skill may create or edit source.local.yaml on this user's behalf — but only when they explicitly ask to become (or stop being) a given source's admin (e.g. "make me the admin for the finance reports source" / "I don't want write access to X anymore"), never as a side effect of just running "index external sources."

How to run this skill

Step 1 — Find connector-backed libs

Scan libs/*/ for a source.yaml. A libs/<name>/ with one is connector-backed and in scope; a libs/<name>/ without one is a git-copy lib and stays completely untouched by this skill.

If no libs/<name>/source.yaml exists anywhere, tell the user there's nothing to index yet and how to add one (a source.yaml with connector and location), and stop — don't fabricate an index from nothing.

Step 2 — Read the local access setting

Read libs/<name>/source.local.yaml if it exists. access: write means this user is an admin for this one source; anything else (the field is read, or the file doesn't exist at all) means read-only. This check is per connector-backed lib — a user can be write-access for one source and read-only for another.

Step 3 — Check the shared index store, and fetch it if it exists

If source.yaml has an index: block, check whether index.location already has a published index (per index.store: a git remote with commits/a reachable ref, or a filesystem/network/shared location with files already in it):

• It exists → fetch/pull it (git pull/clone for git, a plain file copy for a filesystem/network shared path, or the matching connector tool for a shared location that needs one) and use it to refresh libs/<name>/{index.md,entities/,graph/,log.md}. Do this regardless of this user's access level — reading the published index never requires write access, and it's exactly what lets a read-only user skip building anything themselves.
• It's empty / nothing published yet → there's nothing to fetch. Don't treat this as an error; it's the normal first-time state before anyone with write access has run this. Continue to Step 4 — whether that turns into "create it" or "nothing to report" depends entirely on this user's access level.

If no index: block is configured at all, skip this check — the index (if any exists yet) already lives directly in libs/<name>/, same as before.

Step 4 — Decide whether to build or refresh from the live connector

This is the one place access level actually changes behavior:

• Read-only (the common case, and the default for anyone who hasn't set access: write locally): stop here, regardless of what Step 3 found. Never touch the live connector on a read-only user's behalf.
  • If Step 3 fetched something (or one already lived in libs/<name>/ with no index: block): report what it covers.
  • If nothing exists anywhere yet (Step 3 found the shared store empty, or there's no index: block and libs/<name>/ is empty too): say so plainly, and suggest asking whoever administers this source to run it, or setting access: write locally if this user is meant to be that admin.
• Write access: always continue to Step 5, whether Step 3 fetched an existing index (refresh it) or found the store empty (build the very first version from scratch) — the two cases are handled identically from here on; Step 7 is what actually creates the remote copy either way.

Step 5 — Resolve the connector and enumerate/summarize documents

Resolve connector to whatever live tool is actually available this session:

• sharepoint / a Microsoft-hosted location → search (ToolSearch) for a connected Microsoft 365 MCP tool.
• google_drive → search for a connected Google Drive MCP tool.
• web / custom / anything else with a plain URL → WebFetch.

If the needed connector isn't authorized in this session (per the current MCP auth state), don't fail the whole run — tell the user which connector needs authorizing (via claude.ai connector settings, or claude mcp//mcp for non-claude.ai servers), skip that source, and continue with any others.

For each document found at the location:

• Read enough content to write a real 1-2 sentence tldr when the connector allows fetching content.
• If only metadata is available (listing only, no read access), say so honestly in the tldr (e.g. "Metadata only — read access not granted") and set confidence low (e.g. 0.3) rather than fabricating a summary.

Create/update one entity page per document at libs/<name>/entities/<doc-slug>.md:

---
type: document
resource: <document's own URL/path>
tldr: ...
confidence: 0.3–0.9   # per the rule above
quality: ...
last_updated: YYYY-MM-DD
freshness_window_days: 30   # shorter than a typical wiki page — external sources change without notice
retention: medium
---

Step 6 — Update the hub page, graph, index, and log

Create/update libs/<name>/index.md — the root routing page for this connector, mirroring wiki/index.md's own role:

---
type: external-source
resource: <location, from source.yaml>
tldr: One sentence — what this source is and how many documents were found
last_updated: YYYY-MM-DD
---

followed by a short routing table pointing to entities/index.md and graph/index.md.

Record any real relationships between documents (e.g. folder hierarchy) as edges in libs/<name>/graph/edges.json, using a contains/part_of-style edge type — this is fine to be thin or empty for a flat source with no useful structure beyond a document list.

Update libs/<name>/entities/index.md and libs/<name>/graph/index.md (flat bullet lists, no frontmatter, same convention as wiki/entities/ index.md/wiki/graph/index.md). Log every created/updated page in libs/<name>/log.md, same reverse-chronological format as Rule B (CLAUDE.md/AGENTS.md §7) — this log is independent of wiki/log.md; nothing under wiki/ is touched by this skill at all.

Step 7 — Publish, if this source has both write access and a configured index store

If this user has access: write and source.yaml has an index: block, push the refreshed libs/<name>/{index.md,entities/,graph/,log.md} back out to index.location (git push for store: git, a file copy back for a filesystem/network store: shared path, or the matching connector tool for a shared location that needs one) — every single run, not just the first one, so every other user's next fetch (Step 3) picks up the update. This is exactly how a shared index gets created in the first place: if Step 3 found the store empty, this step's push is what populates it for the first time; there's no separate "initialize" action.

If no index: block is configured, there's nothing external to publish; the refreshed files staying inside this repo's own libs/<name>/ (tracked by this repo's own git) is the publish step, same as the original default behavior.

If this user is read-only, this step never runs — nothing is ever pushed or copied to index.location (or, in the no-index:-block case, nothing is even rebuilt in libs/<name>/ to begin with). A read-only user has, by construction, nothing of their own to save back — Step 4 already stopped them before anything was built.

Step 8 — Remind to review and sync

This is always the last step, every time this skill makes any change at all. Close with a short reminder — do not sync or push anything yourself here:

"External source indexing complete for libs/<name>/ — see libs/<name>/log.md for the details. Please review the changes, and once you're happy with them, say 'sync changes' to commit and push them to origin."

If nothing was actually indexed or fetched (no connector-backed libs found, every source was skipped for lack of authorization, or every source was read-only with nothing new to fetch), skip this reminder.

Edge cases

• No connector-backed libs/<name>/ exists — report that and explain how to add one; don't touch anything.
• source.local.yaml doesn't exist yet for a source — treat as read-only; this is the correct default, not a misconfiguration worth flagging loudly. Mention once, briefly, how to become the admin (access: write in that file) if the user seems to expect a rebuild.
• Everyone is read-only and nothing has ever been published — report that clearly rather than silently doing nothing; this is the one case worth flagging as a real gap (nobody can currently index this source).
• Connector not authorized this session — report which one and how to authorize it (claude.ai connector settings / claude mcp / /mcp), skip that source, keep going with the rest. Never ask the user for tokens or callback URLs.
• Fetching a published index (Step 3) fails or the remote is unreachable — report it and fall back to whatever's already cached locally in libs/<name>/ from a previous fetch, rather than blocking the whole run; don't let a transient network issue block a read-only user from seeing the last-known index.
• Re-running against an already-indexed source — refresh existing entity pages in place (update tldr/last_updated/confidence); never duplicate a document's page.
• A previously-indexed document is no longer found at the source — don't delete its page. Update its tldr with a note ("No longer found at source as of YYYY-MM-DD") and set retention: low, so the existing ckb-lint retention sweep archives it naturally on a later pass — no new archival mechanism needed.
• A source is very large — cap what gets fully fetched/summarized in one run and explicitly report what was skipped (e.g. "42 of ~300 documents summarized this pass — re-run to continue"). Never silently truncate while implying full coverage.
• Write access but publishing (Step 7) conflicts with a newer version someone else already pushed — this is why Step 3 (fetch) always runs first, even for write-access users: rebuild on top of the latest fetched state rather than blindly overwriting it. If a real conflict still shows up at publish time, report it rather than force-pushing over it.
• libs/<name>/ has both real content files and a source.yaml — ambiguous and out of scope for this skill to resolve; report it and stop for that folder (this is the same case ckb-lint's external-source config check flags).
• source.yaml is missing connector or location — report it as malformed and skip that source; don't guess at intent.

---

Licensed under the Apache License, Version 2.0 — see LICENSE at the repository root, or http://www.apache.org/licenses/LICENSE-2.0.