Add agents config
This commit is contained in:
Michał Kopeć
parent
c56348b15b
commit
eebfb07df9
33 changed files with 3346 additions and 292 deletions
170
.agents/skills/cascade-kb-init/SKILL.md
Normal file
170
.agents/skills/cascade-kb-init/SKILL.md
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
---
|
||||
name: cascade-kb-init
|
||||
description: Bootstrap a brand-new Cascade Knowledge Base - the same directory structure, AGENTS.md/CLAUDE.md system prompt, and empty wiki/ scaffold as this project - inside a target folder (typically empty, or a new project that doesn't have one yet). Use when the user asks to "set up a new wiki like this one", "initialize a new cascade KB", "bootstrap a wiki here", "create a knowledge base with this schema", or calls it a "wiki initializer". Do not confuse with a generic `init` skill that documents an existing codebase - this one creates the Cascade KB pattern itself, empty, ready for its first ingest.
|
||||
---
|
||||
|
||||
# Cascade KB init skill
|
||||
|
||||
## Purpose
|
||||
|
||||
Copy this project's Cascade Knowledge Base *schema* - not its content - into
|
||||
a new target folder: the directory structure, the `AGENTS.md`/`CLAUDE.md`
|
||||
system prompt that defines how the KB behaves, and the empty `wiki/`
|
||||
scaffold (routing table, overview, log, error book, entity/graph indexes).
|
||||
The result is a new, empty KB that behaves exactly like this one, ready for
|
||||
its first `raw/inbox/` drop and "Ingest."
|
||||
|
||||
This is a one-way copy from this repo's own `AGENTS.md`/`wiki/` template
|
||||
files into a different folder. It never reads or writes anything in this
|
||||
repo's `raw/`, `wiki/entities/`, `wiki/graph/edges.json`, or `outputs/` -
|
||||
those hold this project's actual accumulated knowledge, which is exactly
|
||||
what should *not* travel into a fresh KB.
|
||||
|
||||
## Trigger phrases
|
||||
|
||||
- "set up a new wiki like this one" / "initialize a new cascade KB"
|
||||
- "bootstrap a wiki here" / "create a knowledge base with this schema"
|
||||
- "wiki initializer" / "clone this KB structure into a new project"
|
||||
|
||||
## How to run this skill
|
||||
|
||||
### Step 1 - Confirm the target folder
|
||||
|
||||
Ask (if not already given): "Which folder should I initialize the new
|
||||
knowledge base in?" Resolve to an absolute path. This is a filesystem
|
||||
action outside the current repo, so confirm the resolved path back to the
|
||||
user before writing anything - do not assume a relative path means
|
||||
"somewhere under the current project."
|
||||
|
||||
If the folder doesn't exist yet, create it after confirming the path. If it
|
||||
exists, check its contents before doing anything else (Step 2).
|
||||
|
||||
### Step 2 - Don't clobber an existing KB or unrelated project
|
||||
|
||||
If the target already contains a `wiki/` directory, or an `AGENTS.md` /
|
||||
`CLAUDE.md`, stop and ask: "This folder already looks like it has a
|
||||
knowledge base (found `<what>`). Initializing here could overwrite it. Do
|
||||
you want to proceed anyway, pick a different folder, or only add whatever
|
||||
scaffold pieces are missing?" Never silently overwrite an existing
|
||||
`AGENTS.md` or populated `wiki/` tree.
|
||||
|
||||
If the target has other, unrelated files (e.g. it's an existing code
|
||||
project without a KB yet) - that's fine, proceed; the scaffold is added
|
||||
alongside them. Note any top-level name collisions (e.g. an existing
|
||||
`wiki/` folder used for something else) and ask before touching those
|
||||
specifically.
|
||||
|
||||
### Step 3 - Default skill set (no need to ask)
|
||||
|
||||
The bare scaffold (directory structure + `AGENTS.md`/`CLAUDE.md` + empty
|
||||
`wiki/` templates) is always included, and so is the reusable KB skill set
|
||||
- these operate purely on the `wiki/` structure, so they carry over
|
||||
cleanly and are part of "the schema" as far as this skill is concerned:
|
||||
`export-okf`, `export-starlight`, `sync-changes`, `extract-transcript`,
|
||||
`project-summary`. Don't ask about these - just include them.
|
||||
|
||||
`ghost-writer` and `clouddrift-docx` are not part of the default set (a
|
||||
general writing tool and a brand-specific export skill respectively, not
|
||||
KB-schema-native) - only include either if the user explicitly asks for it,
|
||||
e.g. "also bring over ghost-writer."
|
||||
|
||||
Record the final skill list (default five, plus anything explicitly added)
|
||||
- this affects Steps 6 and 7.
|
||||
|
||||
### Step 4 - Create the directory structure
|
||||
|
||||
Under the target folder, create:
|
||||
|
||||
```
|
||||
libs/
|
||||
linked/
|
||||
outputs/
|
||||
raw/inbox/
|
||||
raw/archive/
|
||||
tmp/
|
||||
wiki/entities/
|
||||
wiki/graph/
|
||||
workload/
|
||||
```
|
||||
|
||||
`libs/`, `linked/`, and `tmp/` are gitignored per the schema (Step 6) and
|
||||
stay empty. `raw/inbox/`, `raw/archive/`, and `workload/` are meant to be
|
||||
tracked but start empty - add a placeholder `.gitkeep` file to each so they
|
||||
survive a fresh `git init` + first commit rather than vanishing as empty
|
||||
directories.
|
||||
|
||||
### Step 5 - Write `AGENTS.md` and the `CLAUDE.md` symlink
|
||||
|
||||
Copy this repo's `AGENTS.md` verbatim into the target as `AGENTS.md` - it
|
||||
is already fully generic (no project-specific content; it *is* the
|
||||
schema definition). Then create `CLAUDE.md` in the target as a symlink to
|
||||
`AGENTS.md`, matching this repo's own convention (one source of truth,
|
||||
readable under either filename).
|
||||
|
||||
### Step 6 - Write the empty `wiki/` scaffold
|
||||
|
||||
Create these files in the target, using this repo's current versions as
|
||||
the template and stripping every reference to this project's actual
|
||||
content (Grant Thornton, Cloud Drift, specific entities, etc.) down to the
|
||||
generic structure:
|
||||
|
||||
- **`wiki/index.md`** - frontmatter with `kb_schema_version: "1.1"` only.
|
||||
Body: the routing table with just its four fixed infrastructure rows
|
||||
(Overview, Log, Error Book, Entities, Graph) and no entity rows, plus the
|
||||
"## Entity Pages" section with its placeholder note. Use today's date
|
||||
where the template needs one.
|
||||
- **`wiki/overview.md`** - copy verbatim from this repo (it's already
|
||||
generic - directory tree, cascade priority, frontmatter summary, no
|
||||
project content). Set `last_updated` to today.
|
||||
- **`wiki/log.md`** - header and explanation only, no entries.
|
||||
- **`wiki/error-book.md`** - copy verbatim (already generic: empty table
|
||||
+ placeholder note). Set `last_updated` to today.
|
||||
- **`wiki/entities/index.md`** - header + placeholder note, no entries.
|
||||
- **`wiki/graph/index.md`** - header + pointer to `edges.json`, with a
|
||||
generic "Current graph coverage: (none yet)" line instead of this
|
||||
repo's specific bullet list.
|
||||
- **`wiki/graph/edges.json`** - `{"version": 1, "last_updated": "<today>", "edges": []}`.
|
||||
|
||||
Do not carry over any entity pages, graph edges, log entries, or overview
|
||||
content specific to this project - the whole point is an empty KB with the
|
||||
same shape.
|
||||
|
||||
### Step 7 - Write `.gitignore`
|
||||
|
||||
Base rules (always): `libs/`, `linked/`, `tmp/`, `.DS_Store`. Since
|
||||
`export-starlight` and `export-okf` are in the default skill set (Step 3),
|
||||
also always add `outputs/starlight` and `outputs/okf` - both exist to be
|
||||
gitignored precisely because those two skills are present by default.
|
||||
|
||||
### Step 8 - Copy the skill set from Step 3
|
||||
|
||||
Copy each skill's folder from this repo's `.agents/skills/<name>/` into the
|
||||
target's `.agents/skills/<name>/` unchanged - the default five, plus
|
||||
anything explicitly added. Then create `.claude/skills` in the target as a
|
||||
symlink to `../.agents/skills`, matching this repo's convention - do this
|
||||
once, after copying the whole set, not per-skill.
|
||||
|
||||
### Step 9 - Report
|
||||
|
||||
Tell the user:
|
||||
- The resolved target path.
|
||||
- The directory tree created.
|
||||
- Whether `AGENTS.md`/`CLAUDE.md` were written or (per Step 2) skipped/merged.
|
||||
- Which skills were copied (the default five, plus anything explicitly added).
|
||||
- Next step: "Drop material into `raw/inbox/` and say 'Ingest' to populate the wiki for the first time."
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **Target is the current repo itself, or a parent/ancestor of it** - refuse
|
||||
and explain why: this would either overwrite the live KB you're running
|
||||
from, or nest a KB inside itself. Ask for a genuinely separate path.
|
||||
- **Target is on a different filesystem/drive or requires elevated
|
||||
permissions** - if directory creation fails, report the exact error
|
||||
rather than retrying with escalated permissions.
|
||||
- **User wants only *some* of the wiki template files** (e.g. just the
|
||||
directory structure, no `AGENTS.md`) - honor that; the steps above are
|
||||
the default full scaffold, not an all-or-nothing bundle.
|
||||
- **This repo's own `AGENTS.md` or template `wiki/` files have since
|
||||
drifted from each other** (e.g. one mentions a directory the other
|
||||
doesn't) - fix the drift in *this* repo first if noticed, then copy the
|
||||
corrected version. Don't propagate a known inconsistency into a new KB.
|
||||
BIN
.agents/skills/clouddrift-docx/.DS_Store
vendored
Normal file
BIN
.agents/skills/clouddrift-docx/.DS_Store
vendored
Normal file
Binary file not shown.
141
.agents/skills/clouddrift-docx/SKILL.md
Normal file
141
.agents/skills/clouddrift-docx/SKILL.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
---
|
||||
name: clouddrift-docx
|
||||
description: Convert any Markdown file into a Cloud Drift branded .docx or .pdf using pandoc and a custom reference template (fonts, colors, logo, footer pagination pulled from Cloud Drift's brand guide and case-study docx). Use when the user asks to "export as a Cloud Drift branded doc/pdf", "convert this markdown to a Cloud Drift docx", "make this look like our case study", or wants a client-ready styled document from a markdown file.
|
||||
---
|
||||
|
||||
# Cloud Drift branded docx/pdf export
|
||||
|
||||
## Purpose
|
||||
|
||||
Turn any Markdown file into a document that looks like it came from Cloud
|
||||
Drift's own brand: Open Sans / Open Sans Light typography, the Fire
|
||||
Opal / White Coffee / Raisin Black color system, the "Cloud Drift." logo in
|
||||
a running header, and page-numbered footers. Output can be `.docx`,
|
||||
`.pdf`, or both.
|
||||
|
||||
The brand spec was extracted directly from `tmp/Cloud_Drift_CI_v1b 2.pdf`
|
||||
(the CI/brand guide) and cross-checked against the real embedded fonts/
|
||||
colors/logo inside `tmp/Branded [case study] ... .docx` — this isn't a
|
||||
guess at "corporate-looking" styling, it replicates the actual brand:
|
||||
|
||||
- **Font:** Open Sans Light (body, title, H1), Open Sans regular/bold (H2/H3, table headers)
|
||||
- **Colors:** Fire Opal `#E45249` (primary/accent — title, H2, hyperlinks, blockquote bar), White Coffee `#E8DCD0` (table shading/borders), Raisin Black `#252525` (body text, H1)
|
||||
- **Logo:** "Cloud Drift." wordmark, placed in the page header on every page
|
||||
- **Layout:** A4, 1" margins, footer with "Page X of Y"
|
||||
|
||||
## How this fits together
|
||||
|
||||
1. `assets/clouddrift-reference.docx` — the pandoc reference-doc. Pandoc
|
||||
clones this file's styles (Normal, Title, Heading 1-6, Block Text,
|
||||
Hyperlink, Table), page setup, and header/footer (including the logo)
|
||||
into whatever it generates. The Open Sans / Open Sans Light fonts are
|
||||
embedded inside it (unobfuscated TTF parts, same scheme the original
|
||||
Cloud Drift case-study docx uses), so Word will render it correctly even
|
||||
on a machine that doesn't have the fonts installed.
|
||||
2. `assets/fonts/*.ttf` — the same 8 font files, for reinstalling the fonts
|
||||
locally if needed (see below).
|
||||
3. `assets/clouddrift-logo.png` — the extracted logo, already baked into
|
||||
the reference doc's header; kept here for reuse elsewhere if needed.
|
||||
4. `scripts/convert.py` — the conversion driver (see usage below).
|
||||
5. `scripts/build_reference.py` + `scripts/embed_fonts.py` — the one-time
|
||||
scripts that built `clouddrift-reference.docx` from pandoc's own default
|
||||
reference doc. Only needed again if the brand changes (new colors, new
|
||||
logo, different heading scheme) — see "Regenerating the template" below.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python3 .claude/skills/clouddrift-docx/scripts/convert.py INPUT.md --format docx
|
||||
python3 .claude/skills/clouddrift-docx/scripts/convert.py INPUT.md --format pdf
|
||||
python3 .claude/skills/clouddrift-docx/scripts/convert.py INPUT.md --format both --output outputs/documents/my-doc
|
||||
```
|
||||
|
||||
- `--format` — `docx` (default), `pdf`, or `both`.
|
||||
- `--output` — path without extension; defaults to the input's own path/name.
|
||||
- `--reference-doc` — override the template (rarely needed).
|
||||
|
||||
The `.docx` step is pure pandoc (`pandoc INPUT.md -o OUTPUT.docx --reference-doc=...`).
|
||||
PDF export works by asking macOS **Pages** to open that generated `.docx`
|
||||
and export it to PDF — this guarantees the PDF is pixel-identical to the
|
||||
branded docx rather than a second, independently-maintained template. This
|
||||
means:
|
||||
- **macOS only.** There's no PDF engine pandoc can drive directly in this
|
||||
environment, so this is the practical path rather than building a
|
||||
parallel LaTeX/CSS template.
|
||||
- **First run may need a permission grant.** macOS will prompt to let the
|
||||
automating process control Pages (System Settings → Privacy & Security →
|
||||
Automation). Approve it once.
|
||||
- **If invoked through Claude Code's Bash tool**, the PDF step needs
|
||||
`dangerouslyDisableSandbox: true` — sandboxed Bash can't send Apple
|
||||
Events to GUI apps like Pages. The docx-only step does not need this.
|
||||
- If Pages returns "Connection is invalid" on the very first call, it
|
||||
usually means Pages hadn't finished launching yet — retry once.
|
||||
|
||||
**Why not `pandoc --pdf-engine=...` directly?** Tried this on 2026-07-14 —
|
||||
installed `tectonic` (a self-contained LaTeX engine) specifically so pandoc
|
||||
could produce PDF natively. It failed: tectonic fetches its TeX resource
|
||||
bundle from `relay.fullyjustified.net` on first use, and that domain
|
||||
resolves to `0.0.0.0` on this network (a DNS-level filter, not something to
|
||||
route around). General internet access otherwise works fine — it's specific
|
||||
to that host. Asked the user how to proceed; they chose to keep the
|
||||
Pages-based PDF path rather than switch to a bigger `BasicTeX` install or
|
||||
allowlist the domain. `tectonic` was uninstalled again. If this is
|
||||
revisited later, either option is viable — see git/workload history for
|
||||
2026-07-14 for the tradeoffs discussed.
|
||||
|
||||
## Font install (one-time, already done as of 2026-07-14)
|
||||
|
||||
Open Sans / Open Sans Light aren't system fonts on macOS by default. They've
|
||||
already been installed to `~/Library/Fonts/` from `assets/fonts/` so Pages/
|
||||
Word render them correctly instead of falling back to a serif substitute.
|
||||
If this is ever run on a different machine, install them first:
|
||||
|
||||
```bash
|
||||
cp .claude/skills/clouddrift-docx/assets/fonts/*.ttf ~/Library/Fonts/
|
||||
```
|
||||
|
||||
(Open Sans is SIL Open Font License — freely redistributable. These exact
|
||||
files came from the Cloud Drift case-study docx in `tmp/`.)
|
||||
|
||||
## Regenerating the template
|
||||
|
||||
Only needed if the brand changes. From a scratch directory:
|
||||
|
||||
```bash
|
||||
pandoc -o pandoc-default-reference.docx --print-default-data-file reference.docx
|
||||
cp <skill>/assets/fonts/*.ttf ./fonts/
|
||||
cp <skill>/assets/clouddrift-logo.png ./clouddrift-logo.png
|
||||
python3 <skill>/scripts/build_reference.py # edit colors/fonts/sizes at the top first if rebranding
|
||||
python3 <skill>/scripts/embed_fonts.py clouddrift-reference.docx
|
||||
cp clouddrift-reference.docx <skill>/assets/clouddrift-reference.docx
|
||||
```
|
||||
|
||||
Then sanity-check visually: convert a test markdown file and render page 1
|
||||
with `qlmanage -t -s 1600 -o <dir> file.pdf` (no poppler/pdftoppm needed) —
|
||||
or install `poppler` (`brew install poppler`) for `pdftoppm` to check
|
||||
arbitrary pages of a multi-page doc, which is what caught the page-break
|
||||
issue below.
|
||||
|
||||
## Known fixes
|
||||
|
||||
- **2026-07-14 — empty page before large tables.** Pandoc's default
|
||||
reference doc sets `keepNext`/`keepLines` on every Heading style
|
||||
(standard Word behavior: never strand a heading alone at the bottom of a
|
||||
page). Under Pages specifically, this backfired when a heading was
|
||||
immediately followed by a large table: the heading got stranded alone on
|
||||
a page and the *entire* table got pushed to the next page, leaving a
|
||||
near-empty page in between. `build_reference.py` now explicitly strips
|
||||
`keepNext`/`keepLines` from Heading 1–9 (see `disable_keep_with_next`) so
|
||||
pagination flows naturally — worst case a heading ends up as the last
|
||||
line on a page, which is a far smaller cosmetic cost than a blank page.
|
||||
If a similar gap ever reappears with some other block type, check the
|
||||
relevant style's `pPr` for `keepNext`/`keepLines`/`pageBreakBefore` first.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- Bullet markers use the default (black) bullet glyph, not the Fire-Opal-red
|
||||
bullet dot seen in the case study — pandoc generates its own numbering
|
||||
definitions per document rather than inheriting the reference doc's, so
|
||||
this isn't controllable through the reference-doc mechanism alone.
|
||||
- PDF export is macOS/Pages-only; there is no cross-platform fallback in
|
||||
this environment.
|
||||
BIN
.agents/skills/clouddrift-docx/assets/clouddrift-logo.png
Normal file
BIN
.agents/skills/clouddrift-docx/assets/clouddrift-logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
BIN
.agents/skills/clouddrift-docx/assets/clouddrift-reference.docx
Normal file
BIN
.agents/skills/clouddrift-docx/assets/clouddrift-reference.docx
Normal file
Binary file not shown.
BIN
.agents/skills/clouddrift-docx/assets/fonts/OpenSans-bold.ttf
Normal file
BIN
.agents/skills/clouddrift-docx/assets/fonts/OpenSans-bold.ttf
Normal file
Binary file not shown.
Binary file not shown.
BIN
.agents/skills/clouddrift-docx/assets/fonts/OpenSans-italic.ttf
Normal file
BIN
.agents/skills/clouddrift-docx/assets/fonts/OpenSans-italic.ttf
Normal file
Binary file not shown.
BIN
.agents/skills/clouddrift-docx/assets/fonts/OpenSans-regular.ttf
Normal file
BIN
.agents/skills/clouddrift-docx/assets/fonts/OpenSans-regular.ttf
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
250
.agents/skills/clouddrift-docx/scripts/build_reference.py
Normal file
250
.agents/skills/clouddrift-docx/scripts/build_reference.py
Normal file
|
|
@ -0,0 +1,250 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Build the Cloud Drift pandoc reference.docx from pandoc's default reference doc."""
|
||||
import copy
|
||||
from docx import Document
|
||||
from docx.shared import Pt, Inches, RGBColor, Emu
|
||||
from docx.enum.text import WD_ALIGN_PARAGRAPH
|
||||
from docx.oxml.ns import qn
|
||||
from docx.oxml import OxmlElement
|
||||
|
||||
SRC = "pandoc-default-reference.docx"
|
||||
OUT = "clouddrift-reference.docx"
|
||||
LOGO = "clouddrift-logo.png"
|
||||
|
||||
FIRE_OPAL = RGBColor(0xE4, 0x52, 0x49)
|
||||
WHITE_COFFEE = RGBColor(0xE8, 0xDC, 0xD0)
|
||||
RAISIN = RGBColor(0x25, 0x25, 0x25)
|
||||
MUTED = RGBColor(0x59, 0x59, 0x59)
|
||||
|
||||
LIGHT = "Open Sans Light"
|
||||
REGULAR = "Open Sans"
|
||||
|
||||
|
||||
def strip_theme_attrs(rpr_or_font_element):
|
||||
"""Remove theme-based color/font attrs so explicit values always win."""
|
||||
if rpr_or_font_element is None:
|
||||
return
|
||||
for tag in ("color",):
|
||||
el = rpr_or_font_element.find(qn(f"w:{tag}"))
|
||||
if el is not None:
|
||||
for attr in ("themeColor", "themeTint", "themeShade"):
|
||||
if el.get(qn(f"w:{attr}")) is not None:
|
||||
del el.attrib[qn(f"w:{attr}")]
|
||||
rfonts = rpr_or_font_element.find(qn("w:rFonts"))
|
||||
if rfonts is not None:
|
||||
for attr in ("asciiTheme", "hAnsiTheme", "eastAsiaTheme", "cstheme"):
|
||||
if rfonts.get(qn(f"w:{attr}")) is not None:
|
||||
del rfonts.attrib[qn(f"w:{attr}")]
|
||||
|
||||
|
||||
def set_style(doc, name, font_name=None, size=None, color=None, bold=None, italic=None):
|
||||
style = doc.styles[name]
|
||||
f = style.font
|
||||
if font_name:
|
||||
f.name = font_name
|
||||
rpr = style.element.get_or_add_rPr()
|
||||
rfonts = rpr.find(qn("w:rFonts"))
|
||||
if rfonts is None:
|
||||
rfonts = OxmlElement("w:rFonts")
|
||||
rpr.append(rfonts)
|
||||
rfonts.set(qn("w:ascii"), font_name)
|
||||
rfonts.set(qn("w:hAnsi"), font_name)
|
||||
rfonts.set(qn("w:cs"), font_name)
|
||||
rfonts.set(qn("w:eastAsia"), font_name)
|
||||
if size:
|
||||
f.size = Pt(size)
|
||||
if color:
|
||||
f.color.rgb = color
|
||||
if bold is not None:
|
||||
f.bold = bold
|
||||
if italic is not None:
|
||||
f.italic = italic
|
||||
strip_theme_attrs(style.element.get_or_add_rPr())
|
||||
|
||||
|
||||
def disable_keep_with_next(doc, names):
|
||||
"""Remove keepNext/keepLines from these styles' pPr.
|
||||
|
||||
Pandoc's default reference doc sets keepNext+keepLines on every Heading
|
||||
style (standard Word behavior: never orphan a heading alone at the
|
||||
bottom of a page). But when the very next block is a large table, this
|
||||
can backfire badly under some renderers (observed in Pages): the
|
||||
heading gets stranded alone on a page and the entire table is pushed to
|
||||
the following page, leaving a mostly-blank page in between. Dropping
|
||||
keepNext/keepLines lets pagination flow naturally instead - worst case
|
||||
a heading ends up as the last line on a page, which is a far smaller
|
||||
cosmetic cost than a near-empty page.
|
||||
"""
|
||||
for name in names:
|
||||
style = doc.styles[name]
|
||||
pPr = style.element.get_or_add_pPr()
|
||||
for tag in ("w:keepNext", "w:keepLines"):
|
||||
el = pPr.find(qn(tag))
|
||||
if el is not None:
|
||||
pPr.remove(el)
|
||||
|
||||
|
||||
def add_field(paragraph, field_code):
|
||||
run = paragraph.add_run()
|
||||
r = run._r
|
||||
fld_begin = OxmlElement("w:fldChar")
|
||||
fld_begin.set(qn("w:fldCharType"), "begin")
|
||||
instr = OxmlElement("w:instrText")
|
||||
instr.set(qn("xml:space"), "preserve")
|
||||
instr.text = f" {field_code} "
|
||||
fld_sep = OxmlElement("w:fldChar")
|
||||
fld_sep.set(qn("w:fldCharType"), "separate")
|
||||
fld_end = OxmlElement("w:fldChar")
|
||||
fld_end.set(qn("w:fldCharType"), "end")
|
||||
r.append(fld_begin)
|
||||
r2 = paragraph.add_run()._r
|
||||
r2.append(instr)
|
||||
r3 = paragraph.add_run()._r
|
||||
r3.append(fld_sep)
|
||||
r4 = paragraph.add_run()._r
|
||||
r4.append(fld_end)
|
||||
|
||||
|
||||
def main():
|
||||
doc = Document(SRC)
|
||||
|
||||
# --- Base body text ---
|
||||
set_style(doc, "Normal", LIGHT, 11, RAISIN)
|
||||
set_style(doc, "Body Text", LIGHT, 11, RAISIN)
|
||||
set_style(doc, "Compact", LIGHT, 11, RAISIN)
|
||||
set_style(doc, "First Paragraph", LIGHT, 11, RAISIN)
|
||||
set_style(doc, "Default Paragraph Font", LIGHT, 11, RAISIN)
|
||||
|
||||
# --- Title / subtitle ---
|
||||
set_style(doc, "Title", LIGHT, 30, FIRE_OPAL, bold=False)
|
||||
doc.styles["Title"].paragraph_format.space_after = Pt(4)
|
||||
set_style(doc, "Subtitle", LIGHT, 14, MUTED, bold=False, italic=False)
|
||||
|
||||
# --- Headings ---
|
||||
set_style(doc, "Heading 1", LIGHT, 22, RAISIN, bold=False)
|
||||
doc.styles["Heading 1"].paragraph_format.space_before = Pt(20)
|
||||
doc.styles["Heading 1"].paragraph_format.space_after = Pt(8)
|
||||
|
||||
set_style(doc, "Heading 2", REGULAR, 16, FIRE_OPAL, bold=True)
|
||||
doc.styles["Heading 2"].paragraph_format.space_before = Pt(16)
|
||||
doc.styles["Heading 2"].paragraph_format.space_after = Pt(6)
|
||||
|
||||
set_style(doc, "Heading 3", REGULAR, 13, RAISIN, bold=True)
|
||||
doc.styles["Heading 3"].paragraph_format.space_before = Pt(12)
|
||||
|
||||
for lvl, sz in ((4, 11.5), (5, 11), (6, 11)):
|
||||
name = f"Heading {lvl}"
|
||||
set_style(doc, name, REGULAR, sz, MUTED, bold=True, italic=(lvl == 6))
|
||||
|
||||
# Avoid huge empty-page gaps when a heading is immediately followed by
|
||||
# a large table (see disable_keep_with_next docstring).
|
||||
disable_keep_with_next(doc, [f"Heading {n}" for n in range(1, 10)])
|
||||
|
||||
# --- Quotes / block text ---
|
||||
set_style(doc, "Block Text", LIGHT, 11, MUTED, italic=True)
|
||||
bt_pPr = doc.styles["Block Text"].element.get_or_add_pPr()
|
||||
pbdr = OxmlElement("w:pBdr")
|
||||
left = OxmlElement("w:left")
|
||||
left.set(qn("w:val"), "single")
|
||||
left.set(qn("w:sz"), "18")
|
||||
left.set(qn("w:space"), "8")
|
||||
left.set(qn("w:color"), "E45249")
|
||||
pbdr.append(left)
|
||||
bt_pPr.append(pbdr)
|
||||
|
||||
# --- Hyperlinks ---
|
||||
set_style(doc, "Hyperlink", LIGHT, None, FIRE_OPAL)
|
||||
hl_rpr = doc.styles["Hyperlink"].element.get_or_add_rPr()
|
||||
u = OxmlElement("w:u")
|
||||
u.set(qn("w:val"), "single")
|
||||
hl_rpr.append(u)
|
||||
|
||||
# --- Table: shaded header row, light borders ---
|
||||
table_style = doc.styles["Table"]
|
||||
tbl_pr = table_style.element.find(qn("w:tblPr"))
|
||||
if tbl_pr is None:
|
||||
tbl_pr = OxmlElement("w:tblPr")
|
||||
table_style.element.append(tbl_pr)
|
||||
borders = OxmlElement("w:tblBorders")
|
||||
for edge in ("top", "left", "bottom", "right", "insideH", "insideV"):
|
||||
el = OxmlElement(f"w:{edge}")
|
||||
el.set(qn("w:val"), "single")
|
||||
el.set(qn("w:sz"), "4")
|
||||
el.set(qn("w:space"), "0")
|
||||
el.set(qn("w:color"), "E8DCD0")
|
||||
borders.append(el)
|
||||
tbl_pr.append(borders)
|
||||
|
||||
style_pr = table_style.element.find(qn("w:tblStylePr"))
|
||||
if style_pr is None:
|
||||
style_pr = OxmlElement("w:tblStylePr")
|
||||
style_pr.set(qn("w:type"), "firstRow")
|
||||
table_style.element.append(style_pr)
|
||||
tc_pr = style_pr.find(qn("w:tcPr"))
|
||||
if tc_pr is None:
|
||||
tc_pr = OxmlElement("w:tcPr")
|
||||
style_pr.append(tc_pr)
|
||||
shd = OxmlElement("w:shd")
|
||||
shd.set(qn("w:val"), "clear")
|
||||
shd.set(qn("w:color"), "auto")
|
||||
shd.set(qn("w:fill"), "E8DCD0")
|
||||
tc_pr.append(shd)
|
||||
rpr_fr = style_pr.find(qn("w:rPr"))
|
||||
if rpr_fr is None:
|
||||
rpr_fr = OxmlElement("w:rPr")
|
||||
style_pr.append(rpr_fr)
|
||||
b_el = OxmlElement("w:b")
|
||||
rpr_fr.append(b_el)
|
||||
color_el = OxmlElement("w:color")
|
||||
color_el.set(qn("w:val"), "252525")
|
||||
rpr_fr.append(color_el)
|
||||
|
||||
# --- Verbatim / code ---
|
||||
try:
|
||||
set_style(doc, "Verbatim Char", None, 10, RAISIN)
|
||||
except KeyError:
|
||||
pass
|
||||
|
||||
# --- Page setup: A4, 1 inch margins ---
|
||||
section = doc.sections[0]
|
||||
section.page_width = Inches(8.27)
|
||||
section.page_height = Inches(11.69)
|
||||
section.left_margin = Inches(1)
|
||||
section.right_margin = Inches(1)
|
||||
section.top_margin = Inches(1)
|
||||
section.bottom_margin = Inches(1)
|
||||
section.header_distance = Inches(0.4)
|
||||
section.footer_distance = Inches(0.4)
|
||||
|
||||
# --- Header: Cloud Drift logo ---
|
||||
header = section.header
|
||||
header.is_linked_to_previous = False
|
||||
hp = header.paragraphs[0]
|
||||
hp.text = ""
|
||||
hp.alignment = WD_ALIGN_PARAGRAPH.LEFT
|
||||
run = hp.add_run()
|
||||
run.add_picture(LOGO, width=Inches(0.85))
|
||||
|
||||
# --- Footer: page number, right aligned, muted ---
|
||||
footer = section.footer
|
||||
footer.is_linked_to_previous = False
|
||||
fp = footer.paragraphs[0]
|
||||
fp.text = ""
|
||||
fp.alignment = WD_ALIGN_PARAGRAPH.RIGHT
|
||||
run = fp.add_run("Page ")
|
||||
run.font.name = REGULAR
|
||||
run.font.size = Pt(9)
|
||||
run.font.color.rgb = MUTED
|
||||
add_field(fp, "PAGE")
|
||||
run2 = fp.add_run(" of ")
|
||||
run2.font.name = REGULAR
|
||||
run2.font.size = Pt(9)
|
||||
run2.font.color.rgb = MUTED
|
||||
add_field(fp, "NUMPAGES")
|
||||
|
||||
doc.save(OUT)
|
||||
print("Saved", OUT)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
112
.agents/skills/clouddrift-docx/scripts/convert.py
Normal file
112
.agents/skills/clouddrift-docx/scripts/convert.py
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Convert a Markdown file into a Cloud Drift branded .docx and/or .pdf.
|
||||
|
||||
Usage:
|
||||
python3 convert.py INPUT.md [--format docx|pdf|both] [--output PATH]
|
||||
[--reference-doc PATH]
|
||||
|
||||
docx is produced by pandoc using the bundled Cloud Drift reference.docx
|
||||
(fonts, colors, logo header, footer pagination). pdf is produced by asking
|
||||
macOS Pages to open that docx and export it to PDF, so the PDF is pixel-for-
|
||||
pixel the same branded layout, not a second independent template.
|
||||
"""
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SKILL_DIR = Path(__file__).resolve().parent.parent
|
||||
REFERENCE_DOC = SKILL_DIR / "assets" / "clouddrift-reference.docx"
|
||||
|
||||
|
||||
def convert_to_docx(input_md: Path, output_docx: Path, reference_doc: Path):
|
||||
if shutil.which("pandoc") is None:
|
||||
sys.exit("pandoc is not installed or not on PATH.")
|
||||
cmd = [
|
||||
"pandoc",
|
||||
str(input_md),
|
||||
"-o",
|
||||
str(output_docx),
|
||||
f"--reference-doc={reference_doc}",
|
||||
"--standalone",
|
||||
]
|
||||
subprocess.run(cmd, check=True)
|
||||
print(f"Wrote {output_docx}")
|
||||
|
||||
|
||||
def convert_docx_to_pdf(input_docx: Path, output_pdf: Path):
|
||||
"""Use macOS Pages (via AppleScript) to export the docx to PDF, preserving
|
||||
the exact branded layout produced by the reference.docx styles."""
|
||||
if sys.platform != "darwin":
|
||||
sys.exit("PDF export currently requires macOS (uses Pages via AppleScript).")
|
||||
script = f'''
|
||||
try
|
||||
tell application "Pages"
|
||||
set theDoc to open POSIX file "{input_docx.resolve()}"
|
||||
delay 2
|
||||
export theDoc to POSIX file "{output_pdf.resolve()}" as PDF
|
||||
close theDoc saving no
|
||||
end tell
|
||||
return "SUCCESS"
|
||||
on error errMsg number errNum
|
||||
return "ERROR: " & errMsg & " (" & errNum & ")"
|
||||
end try
|
||||
'''
|
||||
result = subprocess.run(["osascript", "-e", script], capture_output=True, text=True)
|
||||
out = result.stdout.strip()
|
||||
if out != "SUCCESS":
|
||||
sys.exit(
|
||||
"Pages PDF export failed: "
|
||||
f"{out or result.stderr.strip()}\n"
|
||||
"If this is the first run, macOS may need you to grant automation "
|
||||
"permission for controlling Pages (System Settings > Privacy & "
|
||||
"Security > Automation), or Pages may need a moment after "
|
||||
"launching — try again."
|
||||
)
|
||||
print(f"Wrote {output_pdf}")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("input", type=Path, help="Input Markdown file")
|
||||
parser.add_argument(
|
||||
"--format", choices=["docx", "pdf", "both"], default="docx",
|
||||
help="Output format (default: docx)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output", type=Path, default=None,
|
||||
help="Output path (without extension needed for --format both). "
|
||||
"Defaults to the input filename next to the input file.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--reference-doc", type=Path, default=REFERENCE_DOC,
|
||||
help="Override the Cloud Drift reference.docx template",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.input.exists():
|
||||
sys.exit(f"Input file not found: {args.input}")
|
||||
if not args.reference_doc.exists():
|
||||
sys.exit(f"Reference doc not found: {args.reference_doc}")
|
||||
|
||||
stem = args.output if args.output else args.input.with_suffix("")
|
||||
docx_path = stem.with_suffix(".docx")
|
||||
pdf_path = stem.with_suffix(".pdf")
|
||||
|
||||
if args.format in ("docx", "both"):
|
||||
convert_to_docx(args.input, docx_path, args.reference_doc)
|
||||
|
||||
if args.format in ("pdf", "both"):
|
||||
if not docx_path.exists():
|
||||
convert_to_docx(args.input, docx_path, args.reference_doc)
|
||||
convert_docx_to_pdf(docx_path, pdf_path)
|
||||
if args.format == "pdf" and docx_path.exists() and args.output is None:
|
||||
# pdf-only was requested and we only made the docx as an
|
||||
# intermediate step; clean it up unless the caller named an
|
||||
# explicit --output (in which case leave both, they may want it).
|
||||
docx_path.unlink()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
127
.agents/skills/clouddrift-docx/scripts/embed_fonts.py
Normal file
127
.agents/skills/clouddrift-docx/scripts/embed_fonts.py
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Embed Open Sans / Open Sans Light TTFs into a docx so it renders correctly
|
||||
even on machines that don't have the fonts installed. Mirrors the (unobfuscated)
|
||||
embedding scheme found in the Cloud Drift case-study docx: fontKey all-zeros,
|
||||
plain .ttf parts referenced directly."""
|
||||
import shutil
|
||||
import zipfile
|
||||
import re
|
||||
import sys
|
||||
|
||||
DOCX = sys.argv[1] if len(sys.argv) > 1 else "clouddrift-reference.docx"
|
||||
FONT_DIR = "fonts"
|
||||
|
||||
FONTS = {
|
||||
"Open Sans Light": {
|
||||
"regular": "OpenSansLight-regular.ttf",
|
||||
"bold": "OpenSansLight-bold.ttf",
|
||||
"italic": "OpenSansLight-italic.ttf",
|
||||
"boldItalic": "OpenSansLight-boldItalic.ttf",
|
||||
},
|
||||
"Open Sans": {
|
||||
"regular": "OpenSans-regular.ttf",
|
||||
"bold": "OpenSans-bold.ttf",
|
||||
"italic": "OpenSans-italic.ttf",
|
||||
"boldItalic": "OpenSans-boldItalic.ttf",
|
||||
},
|
||||
}
|
||||
|
||||
NS_R = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
|
||||
FONT_REL_TYPE = f"{NS_R}/font".replace(NS_R, "http://schemas.openxmlformats.org/officeDocument/2006/relationships") + "/font"
|
||||
FONT_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/font"
|
||||
|
||||
|
||||
def build_font_table_and_rels():
|
||||
rid = 1
|
||||
font_entries = []
|
||||
rels = []
|
||||
file_map = {}
|
||||
for name, variants in FONTS.items():
|
||||
embeds = []
|
||||
for kind, fname in variants.items():
|
||||
tag = {
|
||||
"regular": "w:embedRegular",
|
||||
"bold": "w:embedBold",
|
||||
"italic": "w:embedItalic",
|
||||
"boldItalic": "w:embedBoldItalic",
|
||||
}[kind]
|
||||
rId = f"rIdFont{rid}"
|
||||
embeds.append(
|
||||
f'<{tag} r:id="{rId}" w:fontKey="{{00000000-0000-0000-0000-000000000000}}" w:subsetted="0"/>'
|
||||
)
|
||||
rels.append(
|
||||
f'<Relationship Id="{rId}" Type="{FONT_REL_TYPE}" Target="fonts/{fname}"/>'
|
||||
)
|
||||
file_map[rId] = fname
|
||||
rid += 1
|
||||
font_entries.append(f'<w:font w:name="{name}">{"".join(embeds)}</w:font>')
|
||||
|
||||
font_table_xml = (
|
||||
'<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'
|
||||
'<w:fonts xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" '
|
||||
'xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">'
|
||||
+ "".join(font_entries)
|
||||
+ "</w:fonts>"
|
||||
)
|
||||
rels_xml = (
|
||||
'<?xml version="1.0" encoding="UTF-8" standalone="yes"?>'
|
||||
'<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
|
||||
+ "".join(rels)
|
||||
+ "</Relationships>"
|
||||
)
|
||||
return font_table_xml, rels_xml, file_map
|
||||
|
||||
|
||||
def patch_settings(xml_text):
|
||||
if "embedTrueTypeFonts" in xml_text:
|
||||
return xml_text
|
||||
return re.sub(
|
||||
r"(<w:settings[^>]*>)",
|
||||
r'\1<w:embedTrueTypeFonts w:val="1"/>',
|
||||
xml_text,
|
||||
count=1,
|
||||
)
|
||||
|
||||
|
||||
def patch_content_types(xml_text):
|
||||
if 'Extension="ttf"' in xml_text:
|
||||
return xml_text
|
||||
return xml_text.replace(
|
||||
"<Types ",
|
||||
"<Types ",
|
||||
).replace(
|
||||
'<Default Extension="rels"',
|
||||
'<Default Extension="ttf" ContentType="application/x-font-ttf"/><Default Extension="rels"',
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
font_table_xml, rels_xml, file_map = build_font_table_and_rels()
|
||||
|
||||
tmp = DOCX + ".tmp"
|
||||
with zipfile.ZipFile(DOCX, "r") as zin:
|
||||
names = zin.namelist()
|
||||
with zipfile.ZipFile(tmp, "w", zipfile.ZIP_DEFLATED) as zout:
|
||||
for item in zin.infolist():
|
||||
data = zin.read(item.filename)
|
||||
if item.filename == "word/fontTable.xml":
|
||||
data = font_table_xml.encode("utf-8")
|
||||
elif item.filename == "word/settings.xml":
|
||||
data = patch_settings(data.decode("utf-8")).encode("utf-8")
|
||||
elif item.filename == "[Content_Types].xml":
|
||||
data = patch_content_types(data.decode("utf-8")).encode("utf-8")
|
||||
zout.writestr(item, data)
|
||||
|
||||
if "word/_rels/fontTable.xml.rels" not in names:
|
||||
zout.writestr("word/_rels/fontTable.xml.rels", rels_xml)
|
||||
|
||||
for rId, fname in file_map.items():
|
||||
with open(f"{FONT_DIR}/{fname}", "rb") as f:
|
||||
zout.writestr(f"word/fonts/{fname}", f.read())
|
||||
|
||||
shutil.move(tmp, DOCX)
|
||||
print("Embedded fonts into", DOCX)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
147
.agents/skills/export-starlight/SKILL.md
Normal file
147
.agents/skills/export-starlight/SKILL.md
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
---
|
||||
name: export-starlight
|
||||
description: Export the local wiki/ knowledge base into an Astro + Starlight-consumable form at outputs/starlight/, producing a human-readable documentation website. Use when the user asks to "export the wiki to Starlight", "generate the docs site", "export as Astro Starlight", "build a human-readable wiki site", or "publish the knowledge base as a website".
|
||||
---
|
||||
|
||||
# Export to Astro Starlight skill
|
||||
|
||||
## Purpose
|
||||
|
||||
Regenerate `outputs/starlight/` from the current `wiki/` tree so it can be
|
||||
built and served as an Astro + Starlight documentation website — a
|
||||
human-readable, browsable version of this knowledge base (unlike
|
||||
`export-okf`, which targets machine/tool consumption). This is a one-way,
|
||||
on-demand export — `wiki/` stays the authoritative source; `outputs/starlight/`
|
||||
is always a derived artifact of it, never edited by hand and never fed back
|
||||
in.
|
||||
|
||||
This skill only runs when explicitly invoked — it is deliberately not part
|
||||
of the always-loaded `CLAUDE.md`/`AGENTS.md` Ingest/Lint workflows.
|
||||
|
||||
The heavy lifting (frontmatter remapping, link rewriting, asset copying,
|
||||
sidebar generation) is done by a deterministic Python script, not by
|
||||
reading and rewriting every page by hand — the wiki now has 60+ pages, and
|
||||
a mechanical transform like this belongs in code, not in per-page model
|
||||
reasoning. Nothing in this skill needs an LLM to run correctly; the model's
|
||||
job is just to ask the two setup questions below, invoke the script, and
|
||||
relay its report.
|
||||
|
||||
## Trigger phrases
|
||||
|
||||
Use this skill when the user says things like:
|
||||
- "export the wiki to Starlight" / "export as Astro Starlight"
|
||||
- "generate the docs site" / "build the Starlight site"
|
||||
- "publish the knowledge base as a website"
|
||||
- "sync outputs/starlight"
|
||||
|
||||
## How to run this skill
|
||||
|
||||
### Step 1 — Ask the two setup questions, every time
|
||||
|
||||
The user has explicitly said they want to choose these on every invocation
|
||||
— do not assume a default or reuse an answer from a previous run. Use
|
||||
`AskUserQuestion` with:
|
||||
|
||||
1. **Export scope:**
|
||||
- "Full runnable scaffold" → `--mode full` — regenerates the whole
|
||||
`outputs/starlight/` project (package.json, astro.config.mjs with
|
||||
sidebar, tsconfig.json, src/content.config.ts, splash `index.mdx`)
|
||||
plus all content. Fully deletes and rebuilds `outputs/starlight/`.
|
||||
- "Content only" → `--mode content-only` — regenerates only
|
||||
`src/content/docs/**`, `public/assets/**`, and a `SIDEBAR-SNIPPET.md`
|
||||
with the sidebar config to paste in. Leaves any existing scaffold
|
||||
files (package.json, astro.config.mjs, tsconfig.json,
|
||||
content.config.ts, index.mdx) untouched — use this if the user is
|
||||
dropping the output into a Starlight project they already maintain
|
||||
elsewhere.
|
||||
|
||||
2. **Meta pages:**
|
||||
- "Include them" → add `--include-meta` — publishes `wiki/log.md` as a
|
||||
Change Log page and `wiki/error-book.md` as an Error Book page.
|
||||
- "Exclude them" → omit the flag — only entities, sources, graph, and
|
||||
overview get published.
|
||||
|
||||
### Step 2 — Run the script
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/export_starlight.py" --mode <full|content-only> [--include-meta]
|
||||
```
|
||||
|
||||
Resolve `<skill-dir>` to this skill's own directory. Optional extra flag:
|
||||
`--site-title "Custom Title"` overrides the auto-derived site title (which
|
||||
otherwise comes from `wiki/overview.md`'s H1 — a serviceable but generic
|
||||
default like "Knowledge Base Overview"). Consider offering this to the user
|
||||
rather than always silently accepting the default, since it's the one
|
||||
piece of branding a reader sees first.
|
||||
|
||||
### Step 3 — Relay the script's report
|
||||
|
||||
The script prints, and you should summarize back to the user:
|
||||
- Mode used and page count exported.
|
||||
- Assets (images) copied.
|
||||
- Which meta pages were skipped (if `--include-meta` was omitted).
|
||||
- Any unresolved `[[wikilinks]]` (bare wikilinks with no matching page
|
||||
title — these get silently degraded to plain text in the output; a
|
||||
small number is normal wiki-content noise, but flag the list so the user
|
||||
can fix the source page if it's not).
|
||||
- Any broken internal links the validation pass found (a link that was
|
||||
rewritten to a route with no corresponding generated file) — this is a
|
||||
real problem in the source `wiki/` content (a stale or malformed
|
||||
cross-reference), not a bug in the export, and is worth reporting back
|
||||
as something to fix at the source.
|
||||
- A reminder that `outputs/starlight/` has been regenerated on disk but
|
||||
not staged, committed, or `npm install`ed — those are separate steps for
|
||||
the user. For `--mode full`, the site is immediately runnable with
|
||||
`cd outputs/starlight && npm install && npm run dev`.
|
||||
|
||||
## What the transform does (for context, not something you need to re-derive)
|
||||
|
||||
- **Frontmatter:** `tldr` → `description`; the first `# H1` in the body
|
||||
becomes Starlight's required `title` (and is removed from the body, since
|
||||
Starlight renders `title` as the page heading itself — leaving the H1 in
|
||||
would duplicate it). `type`/`confidence`/`quality`/`retention`/
|
||||
`last_updated`/`resource` have no native Starlight field, so they're
|
||||
rendered as a `:::note[Knowledge base metadata]` aside at the top of the
|
||||
page body instead of being dropped.
|
||||
- **Badges:** a page gets a `sidebar.badge` — "Superseded" (if
|
||||
`superseded_by` is set), else "Stale" (if `last_updated` +
|
||||
`freshness_window_days` has elapsed), else "Low confidence" (if
|
||||
`confidence` < 0.5) — priority in that order, one badge max.
|
||||
- **Links:** dual-linked cross-references (`[text](path) ([[Wikilink]])`,
|
||||
or the older `[[Wikilink]] / [text](path)` ordering) collapse to just the
|
||||
markdown link, since Starlight has no wikilink concept. Bare `[[Wikilinks]]`
|
||||
are resolved against every page's H1 title and converted to a proper link
|
||||
where a match exists, otherwise degraded to plain text. Root-absolute
|
||||
`/wiki/...` links become site routes (`/wiki/entities/foo.md` →
|
||||
`/entities/foo/`; a link to a directory's `index.md` maps to the bare
|
||||
directory route, e.g. `/entities/`). `linked/...`/`libs/...` cascade
|
||||
references (pointing outside any self-contained export) become plain
|
||||
text noting the path, not a broken link.
|
||||
- **Assets:** person photos referenced as `../assets/people/x.jpg` are
|
||||
copied to `public/assets/people/x.jpg` and rewritten to `/assets/people/x.jpg`.
|
||||
- **Graph:** `wiki/graph/edges.json` is not copied verbatim (Starlight
|
||||
doesn't render JSON as a page) — its content is fully rendered as a
|
||||
Markdown table appended to `graph/index.md`, sorted by `from`, so no
|
||||
information is lost.
|
||||
- **Sidebar:** one `autogenerate` group per top-level `wiki/` subdirectory
|
||||
(so new pages added to `entities/`/`sources`/`graph`/any future topic
|
||||
folder show up automatically on the next export without touching the
|
||||
sidebar config), plus explicit entries for the handful of root-level
|
||||
pages (Overview, the routing table, and the two optional meta pages).
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **A future `wiki/<newtopic>/` subdirectory:** handled automatically —
|
||||
the script discovers top-level `wiki/` subdirectories dynamically and
|
||||
adds a sidebar `autogenerate` group for each; no script changes needed.
|
||||
- **Re-running with no wiki changes:** should produce byte-identical
|
||||
output (directory listings are sorted before writing). If asset copying
|
||||
or link rewriting ever introduces nondeterminism, that's a bug in the
|
||||
script, not expected behavior.
|
||||
- **A wiki page with no `tldr`:** `description` is simply omitted from
|
||||
that page's frontmatter — Starlight tolerates a missing description.
|
||||
- **A wiki page with no H1:** falls back to a slugified filename as the
|
||||
title (e.g. `foo-bar.md` → "Foo Bar"), same fallback rule `export-okf`
|
||||
uses.
|
||||
573
.agents/skills/export-starlight/scripts/export_starlight.py
Normal file
573
.agents/skills/export-starlight/scripts/export_starlight.py
Normal file
|
|
@ -0,0 +1,573 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Export wiki/ into an Astro + Starlight-consumable form at outputs/starlight/.
|
||||
|
||||
Usage:
|
||||
python3 export_starlight.py --mode {full,content-only} [--include-meta] [--today YYYY-MM-DD]
|
||||
|
||||
Run from the repository root (the directory containing wiki/ and outputs/).
|
||||
|
||||
--mode full regenerate the whole outputs/starlight/ project: scaffold
|
||||
(package.json, astro.config.mjs, tsconfig.json,
|
||||
src/content.config.ts, splash index.mdx) plus content.
|
||||
--mode content-only regenerate only src/content/docs/** and public/assets/**
|
||||
(and a SIDEBAR-SNIPPET.md), leaving any existing scaffold
|
||||
files untouched. Use this if dropping the output into a
|
||||
Starlight project you already have.
|
||||
--include-meta also publish wiki/log.md (as Change Log) and
|
||||
wiki/error-book.md (as Error Book). Omit to keep those
|
||||
internal-only.
|
||||
--today override "today" for staleness checks (default: real
|
||||
today). Mainly useful for reproducible test runs.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path.cwd()
|
||||
WIKI = REPO_ROOT / "wiki"
|
||||
OUT = REPO_ROOT / "outputs" / "starlight"
|
||||
|
||||
RESERVED_ROOT_ROUTES = {
|
||||
"index.md": "routing-table",
|
||||
"overview.md": "overview",
|
||||
"log.md": "changelog",
|
||||
"error-book.md": "error-book",
|
||||
}
|
||||
|
||||
|
||||
def yaml_quote(s):
|
||||
s = str(s).replace("\\", "\\\\").replace('"', '\\"')
|
||||
return f'"{s}"'
|
||||
|
||||
|
||||
def parse_frontmatter(text):
|
||||
"""Minimal parser for this repo's flat key: value frontmatter (no lists/nesting)."""
|
||||
if not text.startswith("---\n"):
|
||||
return {}, text
|
||||
end = text.find("\n---", 4)
|
||||
if end == -1:
|
||||
return {}, text
|
||||
fm_block = text[4:end]
|
||||
body = text[end + 4 :].lstrip("\n")
|
||||
fm = {}
|
||||
for line in fm_block.splitlines():
|
||||
line = line.strip()
|
||||
if not line or ":" not in line:
|
||||
continue
|
||||
key, _, value = line.partition(":")
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
|
||||
value = value[1:-1]
|
||||
fm[key] = value
|
||||
return fm, body
|
||||
|
||||
|
||||
def extract_h1(body):
|
||||
lines = body.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("# "):
|
||||
title = stripped[2:].strip()
|
||||
remaining = lines[:i] + lines[i + 1 :]
|
||||
# drop one immediately-following blank line so we don't leave a gap
|
||||
if i < len(remaining) and remaining[i].strip() == "":
|
||||
remaining = remaining[:i] + remaining[i + 1 :]
|
||||
return title, "\n".join(remaining).lstrip("\n")
|
||||
if stripped:
|
||||
break # first non-blank line isn't an H1 - stop looking
|
||||
return None, body
|
||||
|
||||
|
||||
def wiki_relpath_to_route(relpath):
|
||||
"""relpath is a path string relative to wiki/, e.g. 'entities/adam-betts.md'."""
|
||||
relpath = relpath.lstrip("/")
|
||||
if relpath in RESERVED_ROOT_ROUTES:
|
||||
return "/" + RESERVED_ROOT_ROUTES[relpath] + "/"
|
||||
p = Path(relpath)
|
||||
parts = list(p.parts)
|
||||
if parts and parts[-1].endswith(".md"):
|
||||
parts[-1] = parts[-1][:-3]
|
||||
if parts and parts[-1] == "index":
|
||||
parts = parts[:-1]
|
||||
if not parts:
|
||||
return "/"
|
||||
return "/" + "/".join(parts) + "/"
|
||||
|
||||
|
||||
def out_path_for(relpath):
|
||||
"""Where a wiki/<relpath> file's exported page lives under src/content/docs/."""
|
||||
if relpath in RESERVED_ROOT_ROUTES:
|
||||
return Path(RESERVED_ROOT_ROUTES[relpath] + ".md")
|
||||
return Path(relpath)
|
||||
|
||||
|
||||
def collect_wiki_pages():
|
||||
pages = [] # list of (relpath_str, absolute_path)
|
||||
for path in sorted(WIKI.rglob("*.md")):
|
||||
relpath = path.relative_to(WIKI).as_posix()
|
||||
pages.append((relpath, path))
|
||||
return pages
|
||||
|
||||
|
||||
def build_title_index(pages):
|
||||
"""title -> route, for resolving bare [[Wikilinks]]."""
|
||||
index = {}
|
||||
for relpath, path in pages:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
_, body = parse_frontmatter(text)
|
||||
title, _ = extract_h1(body)
|
||||
route = wiki_relpath_to_route(relpath)
|
||||
if title:
|
||||
index.setdefault(title, route)
|
||||
index.setdefault(title.lower(), route)
|
||||
slug = re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")
|
||||
if slug:
|
||||
index.setdefault(slug, route)
|
||||
rel_no_ext = relpath[:-3] if relpath.endswith(".md") else relpath
|
||||
index.setdefault(rel_no_ext, route)
|
||||
index.setdefault(rel_no_ext.lower(), route)
|
||||
stem = Path(relpath).stem
|
||||
if stem != "index":
|
||||
index.setdefault(stem, route)
|
||||
index.setdefault(stem.lower(), route)
|
||||
return index
|
||||
|
||||
|
||||
WIKILINK_ADJACENT_RE = re.compile(r"(\[[^\]]+\]\([^)]+\))\s*\(\[\[[^\]]+\]\]\)")
|
||||
WIKILINK_BEFORE_LINK_RE = re.compile(r"\[\[[^\]]+\]\]\s*/\s*(\[[^\]]+\]\([^)]+\))")
|
||||
BARE_WIKILINK_RE = re.compile(r"\[\[([^\]]+)\]\]")
|
||||
WIKI_ABS_NON_MD_LINK_RE = re.compile(r"\[([^\]]+)\]\(/wiki/([^)#]+)(#[^)]*)?\)")
|
||||
WIKI_ABS_LINK_RE = re.compile(r"\]\(/wiki/([^)#]+)(#[^)]*)?\)")
|
||||
CASCADE_LINK_RE = re.compile(r"\[([^\]]+)\]\((?:linked|libs)/[^)]+\)")
|
||||
IMAGE_REL_RE = re.compile(r"!\[([^\]]*)\]\(\.\./assets/([^)]+)\)")
|
||||
|
||||
|
||||
def rewrite_body(body, title_index, unresolved_wikilinks, copied_assets):
|
||||
# 1. strip wikilinks that ride alongside a markdown link (either order:
|
||||
# "[text](path) ([[Wikilink]])" or the legacy "[[Wikilink]] / [text](path)")
|
||||
body = WIKILINK_ADJACENT_RE.sub(r"\1", body)
|
||||
body = WIKILINK_BEFORE_LINK_RE.sub(r"\1", body)
|
||||
|
||||
# 2. resolve (or strip) any remaining bare [[Wikilinks]]
|
||||
def _bare(m):
|
||||
text = m.group(1)
|
||||
route = title_index.get(text) or title_index.get(text.lower())
|
||||
if route:
|
||||
return f"[{text}]({route})"
|
||||
unresolved_wikilinks.append(text)
|
||||
return text
|
||||
|
||||
body = BARE_WIKILINK_RE.sub(_bare, body)
|
||||
|
||||
# 3. defuse /wiki/... links to non-Markdown source files not exported as pages
|
||||
def _abs_non_md_link(m):
|
||||
text = m.group(1)
|
||||
relpath = m.group(2)
|
||||
if relpath.endswith(".md"):
|
||||
return m.group(0)
|
||||
if relpath == "graph/edges.json":
|
||||
return f"`{text}` (rendered below)"
|
||||
return f"{text} (source file: `/wiki/{relpath}`)"
|
||||
|
||||
body = WIKI_ABS_NON_MD_LINK_RE.sub(_abs_non_md_link, body)
|
||||
|
||||
# 4. rewrite /wiki/... absolute links to site routes
|
||||
def _abs_link(m):
|
||||
relpath = m.group(1)
|
||||
frag = m.group(2) or ""
|
||||
route = wiki_relpath_to_route(relpath)
|
||||
if frag:
|
||||
route = route.rstrip("/") + "/" + frag
|
||||
return f"]({route})"
|
||||
|
||||
body = WIKI_ABS_LINK_RE.sub(_abs_link, body)
|
||||
|
||||
# 5. defuse linked/ and libs/ cascade references (not part of this export)
|
||||
def _cascade(m):
|
||||
text, target = m.group(1), m.group(0)
|
||||
path = re.search(r"\((?:linked|libs)/([^)]+)\)", target).group(0)[1:-1]
|
||||
return f"{text} (upstream reference: `{path}`, not included in this export)"
|
||||
|
||||
body = CASCADE_LINK_RE.sub(_cascade, body)
|
||||
|
||||
# 6. copy relative person/asset images and rewrite to /assets/...
|
||||
def _image(m):
|
||||
alt, rel = m.group(1), m.group(2)
|
||||
src = WIKI / "assets" / rel
|
||||
if src.is_file():
|
||||
dest = OUT / "public" / "assets" / rel
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(src, dest)
|
||||
copied_assets.append(rel)
|
||||
return f""
|
||||
|
||||
body = IMAGE_REL_RE.sub(_image, body)
|
||||
|
||||
return body
|
||||
|
||||
|
||||
def build_metadata_block(fm, today):
|
||||
lines = []
|
||||
meta_bits = []
|
||||
for label, key in (("Type", "type"), ("Confidence", "confidence"),
|
||||
("Quality", "quality"), ("Last updated", "last_updated"),
|
||||
("Retention", "retention")):
|
||||
if fm.get(key):
|
||||
meta_bits.append(f"**{label}:** {fm[key]}")
|
||||
if meta_bits:
|
||||
lines.append(":::note[Knowledge base metadata]")
|
||||
lines.append(" · ".join(meta_bits))
|
||||
lines.append(":::")
|
||||
lines.append("")
|
||||
|
||||
if fm.get("resource"):
|
||||
resource = fm["resource"]
|
||||
if resource.startswith("http://") or resource.startswith("https://"):
|
||||
lines.append(f":::note[Source]\n[{resource}]({resource})\n:::\n")
|
||||
else:
|
||||
lines.append(f":::note[Source]\n`{resource}`\n:::\n")
|
||||
|
||||
if fm.get("superseded_by"):
|
||||
lines.append(
|
||||
f":::caution[Superseded]\nThis page is superseded by "
|
||||
f"`{fm['superseded_by']}`.\n:::\n"
|
||||
)
|
||||
if fm.get("supersedes"):
|
||||
lines.append(
|
||||
f":::tip[Supersedes]\nThis page supersedes `{fm['supersedes']}`.\n:::\n"
|
||||
)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def compute_badge(fm, today):
|
||||
if fm.get("superseded_by"):
|
||||
return {"text": "Superseded", "variant": "danger"}
|
||||
try:
|
||||
if fm.get("last_updated") and fm.get("freshness_window_days"):
|
||||
last_updated = date.fromisoformat(fm["last_updated"])
|
||||
window = int(fm["freshness_window_days"])
|
||||
if (today - last_updated).days > window:
|
||||
return {"text": "Stale", "variant": "caution"}
|
||||
except ValueError:
|
||||
pass
|
||||
try:
|
||||
if fm.get("confidence") and float(fm["confidence"]) < 0.5:
|
||||
return {"text": "Low confidence", "variant": "note"}
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def render_page(relpath, path, title_index, today, report):
|
||||
text = path.read_text(encoding="utf-8")
|
||||
fm, body = parse_frontmatter(text)
|
||||
h1_title, body = extract_h1(body)
|
||||
title = h1_title or Path(relpath).stem.replace("-", " ").title()
|
||||
|
||||
unresolved = []
|
||||
body = rewrite_body(body, title_index, unresolved, report["copied_assets"])
|
||||
report["unresolved_wikilinks"].extend(
|
||||
f"{relpath}: [[{t}]]" for t in unresolved
|
||||
)
|
||||
|
||||
meta_block = build_metadata_block(fm, today)
|
||||
badge = compute_badge(fm, today)
|
||||
|
||||
fm_lines = ["---", f"title: {yaml_quote(title)}"]
|
||||
if fm.get("tldr"):
|
||||
fm_lines.append(f"description: {yaml_quote(fm['tldr'])}")
|
||||
if badge:
|
||||
fm_lines.append("sidebar:")
|
||||
fm_lines.append(" badge:")
|
||||
fm_lines.append(f" text: {yaml_quote(badge['text'])}")
|
||||
fm_lines.append(f" variant: {badge['variant']}")
|
||||
fm_lines.append("---")
|
||||
|
||||
out_body = (meta_block + "\n" + body) if meta_block else body
|
||||
return "\n".join(fm_lines) + "\n\n" + out_body.strip() + "\n"
|
||||
|
||||
|
||||
def normalize_edges(raw_edges):
|
||||
if isinstance(raw_edges, dict):
|
||||
raw_edges = raw_edges.get("edges", [])
|
||||
if not isinstance(raw_edges, list):
|
||||
return []
|
||||
|
||||
normalized = []
|
||||
for edge in raw_edges:
|
||||
if not isinstance(edge, dict):
|
||||
continue
|
||||
normalized.append(
|
||||
{
|
||||
"from": edge.get("from") or edge.get("source") or "",
|
||||
"type": edge.get("type") or edge.get("relationship") or "",
|
||||
"to": edge.get("to") or edge.get("target") or "",
|
||||
"note": edge.get("note") or edge.get("evidence") or "",
|
||||
"source_files": edge.get("source_files") or [],
|
||||
}
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def table_cell(value):
|
||||
return str(value).replace("|", "\\|").replace("\n", " ")
|
||||
|
||||
|
||||
def render_edges_table(edges):
|
||||
edges = normalize_edges(edges)
|
||||
if not edges:
|
||||
return "\n*No edges recorded yet.*\n"
|
||||
rows = sorted(edges, key=lambda e: (e.get("from", ""), e.get("type", "")))
|
||||
lines = ["", "## All relationships", "",
|
||||
"| From | Relationship | To | Evidence | Sources |", "|---|---|---|---|---|"]
|
||||
for e in rows:
|
||||
sources = ", ".join(f"`{table_cell(s)}`" for s in e.get("source_files", []))
|
||||
lines.append(
|
||||
f"| `{table_cell(e.get('from', ''))}` | {table_cell(e.get('type', ''))} | "
|
||||
f"`{table_cell(e.get('to', ''))}` | {table_cell(e.get('note', ''))} | {sources} |"
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def derive_site_title():
|
||||
overview = WIKI / "overview.md"
|
||||
if overview.is_file():
|
||||
_, body = parse_frontmatter(overview.read_text(encoding="utf-8"))
|
||||
title, _ = extract_h1(body)
|
||||
if title:
|
||||
return title
|
||||
return REPO_ROOT.name
|
||||
|
||||
|
||||
SPLASH_TEMPLATE = """---
|
||||
title: {title}
|
||||
description: {description}
|
||||
template: splash
|
||||
hero:
|
||||
tagline: {tagline}
|
||||
actions:
|
||||
- text: Browse entities
|
||||
link: /entities/
|
||||
icon: right-arrow
|
||||
- text: Browse sources
|
||||
link: /sources/
|
||||
icon: right-arrow
|
||||
- text: Knowledge base index
|
||||
link: /routing-table/
|
||||
icon: document
|
||||
variant: minimal
|
||||
---
|
||||
|
||||
import {{ CardGrid, Card }} from '@astrojs/starlight/components';
|
||||
|
||||
<CardGrid>
|
||||
\t<Card title="Entities" icon="document">
|
||||
\t\tPeople, organisations, products and concepts tracked in this knowledge base.
|
||||
\t</Card>
|
||||
\t<Card title="Sources" icon="open-book">
|
||||
\t\tOne-page summaries of ingested raw material, with provenance back to the original source.
|
||||
\t</Card>
|
||||
\t<Card title="Graph" icon="random">
|
||||
\t\tTyped relationships between entities and sources - who works for whom, what depends on what, what contradicts what.
|
||||
\t</Card>
|
||||
</CardGrid>
|
||||
"""
|
||||
|
||||
PACKAGE_JSON_TEMPLATE = """{{
|
||||
"name": "{name}",
|
||||
"type": "module",
|
||||
"version": "0.0.1",
|
||||
"scripts": {{
|
||||
"dev": "astro dev",
|
||||
"start": "astro dev",
|
||||
"build": "astro build",
|
||||
"preview": "astro preview"
|
||||
}},
|
||||
"dependencies": {{
|
||||
"@astrojs/starlight": "^0.41.3",
|
||||
"astro": "^7.0.9"
|
||||
}}
|
||||
}}
|
||||
"""
|
||||
|
||||
TSCONFIG_TEMPLATE = """{
|
||||
"extends": "astro/tsconfigs/strict"
|
||||
}
|
||||
"""
|
||||
|
||||
CONTENT_CONFIG_TEMPLATE = """import { defineCollection } from 'astro:content';
|
||||
import { docsLoader } from '@astrojs/starlight/loaders';
|
||||
import { docsSchema } from '@astrojs/starlight/schema';
|
||||
|
||||
export const collections = {
|
||||
docs: defineCollection({ loader: docsLoader(), schema: docsSchema() }),
|
||||
};
|
||||
"""
|
||||
|
||||
|
||||
def build_sidebar_js(top_dirs, include_meta):
|
||||
items = []
|
||||
items.append(" { label: 'Knowledge Base Index', slug: 'routing-table' },")
|
||||
items.append(" { label: 'Overview', slug: 'overview' },")
|
||||
if include_meta:
|
||||
items.append(" { label: 'Error Book', slug: 'error-book' },")
|
||||
items.append(" { label: 'Change Log', slug: 'changelog' },")
|
||||
for d in sorted(top_dirs):
|
||||
label = d.replace("-", " ").title()
|
||||
items.append(
|
||||
f" {{ label: '{label}', items: [{{ autogenerate: {{ directory: '{d}' }} }}] }},"
|
||||
)
|
||||
return "[\n" + "\n".join(items) + "\n ]"
|
||||
|
||||
|
||||
def astro_config_template(site_title, top_dirs, include_meta):
|
||||
sidebar_js = build_sidebar_js(top_dirs, include_meta)
|
||||
return f"""import {{ defineConfig }} from 'astro/config';
|
||||
import starlight from '@astrojs/starlight';
|
||||
|
||||
export default defineConfig({{
|
||||
integrations: [
|
||||
starlight({{
|
||||
title: {yaml_quote(site_title)},
|
||||
sidebar: {sidebar_js},
|
||||
}}),
|
||||
],
|
||||
}});
|
||||
"""
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--mode", choices=["full", "content-only"], required=True)
|
||||
parser.add_argument("--include-meta", action="store_true")
|
||||
parser.add_argument("--today", default=None)
|
||||
parser.add_argument(
|
||||
"--site-title", default=None,
|
||||
help="Override the site title (default: derived from wiki/overview.md's H1)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if not WIKI.is_dir():
|
||||
sys.exit(f"No wiki/ directory found under {REPO_ROOT} - run this from the repo root.")
|
||||
|
||||
today = date.fromisoformat(args.today) if args.today else date.today()
|
||||
|
||||
pages = collect_wiki_pages()
|
||||
title_index = build_title_index(pages)
|
||||
|
||||
docs_dir = OUT / "src" / "content" / "docs"
|
||||
assets_dir = OUT / "public" / "assets"
|
||||
|
||||
if args.mode == "full" and OUT.exists():
|
||||
shutil.rmtree(OUT)
|
||||
else:
|
||||
if docs_dir.exists():
|
||||
shutil.rmtree(docs_dir)
|
||||
if assets_dir.exists():
|
||||
shutil.rmtree(assets_dir)
|
||||
docs_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
report = {"copied_assets": [], "unresolved_wikilinks": [], "pages": 0, "skipped_meta": []}
|
||||
|
||||
top_dirs = set()
|
||||
edges_path = WIKI / "graph" / "edges.json"
|
||||
edges = json.loads(edges_path.read_text(encoding="utf-8")) if edges_path.is_file() else []
|
||||
|
||||
for relpath, path in pages:
|
||||
if relpath in ("log.md", "error-book.md") and not args.include_meta:
|
||||
report["skipped_meta"].append(relpath)
|
||||
continue
|
||||
|
||||
rendered = render_page(relpath, path, title_index, today, report)
|
||||
|
||||
if relpath == "graph/index.md":
|
||||
rendered = rendered.rstrip("\n") + "\n" + render_edges_table(edges)
|
||||
|
||||
dest = docs_dir / out_path_for(relpath)
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
dest.write_text(rendered, encoding="utf-8")
|
||||
report["pages"] += 1
|
||||
|
||||
parts = Path(relpath).parts
|
||||
if len(parts) > 1:
|
||||
top_dirs.add(parts[0])
|
||||
|
||||
site_title = args.site_title or derive_site_title()
|
||||
|
||||
if args.mode == "full":
|
||||
(OUT / "package.json").write_text(
|
||||
PACKAGE_JSON_TEMPLATE.format(name=re.sub(r"[^a-z0-9-]", "-", site_title.lower())),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(OUT / "tsconfig.json").write_text(TSCONFIG_TEMPLATE, encoding="utf-8")
|
||||
(OUT / "src" / "content.config.ts").write_text(CONTENT_CONFIG_TEMPLATE, encoding="utf-8")
|
||||
(OUT / "astro.config.mjs").write_text(
|
||||
astro_config_template(site_title, top_dirs, args.include_meta), encoding="utf-8"
|
||||
)
|
||||
overview_fm, overview_body = ({}, "")
|
||||
overview_path = WIKI / "overview.md"
|
||||
if overview_path.is_file():
|
||||
overview_fm, overview_body = parse_frontmatter(overview_path.read_text(encoding="utf-8"))
|
||||
tagline = overview_fm.get("tldr", "A cascade knowledge base.")
|
||||
(docs_dir / "index.mdx").write_text(
|
||||
SPLASH_TEMPLATE.format(
|
||||
title=yaml_quote(site_title),
|
||||
description=yaml_quote(tagline),
|
||||
tagline=yaml_quote(tagline),
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
else:
|
||||
snippet = (
|
||||
"# Sidebar snippet\n\n"
|
||||
"Paste this into your own `astro.config.mjs`, inside "
|
||||
"`starlight({ ... })`:\n\n"
|
||||
"```js\n"
|
||||
f"sidebar: {build_sidebar_js(top_dirs, args.include_meta)},\n"
|
||||
"```\n\n"
|
||||
"Copy `src/content/docs/**` into your project's equivalent "
|
||||
"directory, and `public/assets/**` into your project's "
|
||||
"`public/assets/`.\n"
|
||||
)
|
||||
(OUT / "SIDEBAR-SNIPPET.md").write_text(snippet, encoding="utf-8")
|
||||
|
||||
# Validate internal links resolve to a generated file
|
||||
broken = []
|
||||
for relpath, _ in pages:
|
||||
if relpath in ("log.md", "error-book.md") and not args.include_meta:
|
||||
continue
|
||||
dest = docs_dir / out_path_for(relpath)
|
||||
text = dest.read_text(encoding="utf-8")
|
||||
for m in re.finditer(r"\]\((/[a-zA-Z0-9_\-./]+/)\)", text):
|
||||
route = m.group(1)
|
||||
candidate_dir = docs_dir / route.strip("/")
|
||||
candidate_file = docs_dir / (route.strip("/") + ".md")
|
||||
candidate_index = candidate_dir / "index.md"
|
||||
candidate_mdx = docs_dir / (route.strip("/") + ".mdx") if route.strip("/") else docs_dir / "index.mdx"
|
||||
if not (candidate_file.is_file() or candidate_index.is_file() or candidate_mdx.is_file()):
|
||||
broken.append(f"{relpath} -> {route}")
|
||||
|
||||
print(f"mode: {args.mode}")
|
||||
print(f"pages exported: {report['pages']}")
|
||||
print(f"assets copied: {len(report['copied_assets'])}")
|
||||
if report["skipped_meta"]:
|
||||
print(f"meta pages skipped (--include-meta not set): {', '.join(report['skipped_meta'])}")
|
||||
if report["unresolved_wikilinks"]:
|
||||
print(f"unresolved [[wikilinks]] (stripped to plain text): {len(report['unresolved_wikilinks'])}")
|
||||
for w in report["unresolved_wikilinks"]:
|
||||
print(f" - {w}")
|
||||
if broken:
|
||||
print(f"WARNING: {len(broken)} internal link(s) do not resolve to a generated page:")
|
||||
for b in broken:
|
||||
print(f" - {b}")
|
||||
print(f"output: {OUT}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
84
.agents/skills/extract-transcript/SKILL.md
Normal file
84
.agents/skills/extract-transcript/SKILL.md
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
---
|
||||
name: extract-transcript
|
||||
description: Extract the meeting transcript from a saved Fireflies.ai (or similar) .mhtml page capture into a Markdown file. Use when the user asks to "extract the transcript", "convert this mhtml to a transcript", "get the transcript out of this mhtml file", or provides a `.mhtml` meeting recording file and wants the transcript pulled out. Takes one filename argument.
|
||||
---
|
||||
|
||||
# Extract transcript skill
|
||||
|
||||
## Purpose
|
||||
|
||||
Fireflies.ai (and similar tools) render meeting transcripts client-side,
|
||||
behind login - `WebFetch` and plain `curl`/headless-browser scraping only
|
||||
see the app shell, never the transcript. The reliable workaround is for the
|
||||
user to save the fully-rendered meeting page as a browser `.mhtml` page
|
||||
capture (File > Save Page As > Webpage, Single File, or the browser's
|
||||
"Save as MHTML" option). This skill turns that `.mhtml` capture into a
|
||||
clean Markdown transcript, deterministically - no model reasoning needed.
|
||||
|
||||
## How to run this skill
|
||||
|
||||
Run the bundled script with Bash, passing the `.mhtml` file path as the
|
||||
only argument:
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/extract_transcript.py" "<path/to/file.mhtml>"
|
||||
```
|
||||
|
||||
Resolve `<skill-dir>` to this skill's own directory (the directory
|
||||
containing this SKILL.md) and `<path/to/file.mhtml>` to the file the user
|
||||
gave you - accept it whether they pasted an absolute path, a relative
|
||||
path, or just referenced an open/attached file.
|
||||
|
||||
The script:
|
||||
- Parses the `.mhtml` as a MIME multipart message (Python's `email`
|
||||
module) and finds the rendered HTML part(s).
|
||||
- Locates the transcript panel (`id` ending in `content-transcript`, or
|
||||
falls back to the largest `ScrollArea-styled__Root` element) and walks
|
||||
its structured DOM - per-paragraph speaker name, timestamp, and sentence
|
||||
text - rather than flattening all text, which would lose speaker
|
||||
boundaries.
|
||||
- Extracts the page `<title>` and a best-effort meeting date/time from the
|
||||
page text.
|
||||
- Writes the output next to the input file, same directory and basename,
|
||||
with a `.md` extension (e.g. `Team sync.mhtml` -> `Team sync.md`),
|
||||
overwriting any existing file at that path.
|
||||
|
||||
If `beautifulsoup4` isn't installed, the script exits with the exact
|
||||
`pip3 install beautifulsoup4` command to run - run it, then retry.
|
||||
|
||||
## After running
|
||||
|
||||
Report the output path and the number of transcript lines extracted (the
|
||||
script prints both). Do not summarize or otherwise process the transcript
|
||||
unless the user separately asks for that (e.g. ingesting it into the
|
||||
wiki) - this skill's job ends at producing the `.md` file.
|
||||
|
||||
The script also detects and reports two common capture mistakes rather
|
||||
than silently producing bad output:
|
||||
|
||||
- **Wrong tab active:** if the page was saved while a tab other than
|
||||
Transcript was open (commonly Notes), Fireflies never mounted the
|
||||
transcript panel into the DOM at all. The script detects this and tells
|
||||
the user to reopen the meeting, click Transcript, and re-save.
|
||||
- **Incomplete scroll:** Fireflies virtualizes the transcript list, so if
|
||||
the user didn't scroll all the way through it before saving, only the
|
||||
visible portion is captured and the rest is silently missing (not a
|
||||
quiet stretch of the meeting). The script still writes the `.md` file in
|
||||
this case but prints a warning to stderr for any gap of 90+ seconds
|
||||
between consecutive lines, with the exact timestamp range of each gap,
|
||||
and the same "scroll to the end, then re-save" guidance. Pass this
|
||||
warning on to the user rather than treating a successful "Wrote N lines"
|
||||
message as automatically complete.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Built against Fireflies.ai's current page structure (styled-components
|
||||
class names change on redeploys, so a Fireflies UI change could break
|
||||
the selectors - if extraction fails, inspect the `.mhtml`'s HTML part
|
||||
for the new transcript container structure and update
|
||||
`scripts/extract_transcript.py` accordingly).
|
||||
- Only captures speakers and sentences visible in the saved page. If the
|
||||
transcript panel wasn't fully scrolled/loaded before saving, only the
|
||||
loaded portion will be present.
|
||||
- Attendee list is inferred purely from who has transcript lines - silent
|
||||
attendees who never spoke won't appear.
|
||||
233
.agents/skills/extract-transcript/scripts/extract_transcript.py
Normal file
233
.agents/skills/extract-transcript/scripts/extract_transcript.py
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Extract a Fireflies.ai (or similar) meeting transcript from a saved .mhtml page.
|
||||
|
||||
Usage:
|
||||
python3 extract_transcript.py <path/to/page.mhtml>
|
||||
|
||||
Writes <path/to/page>.md next to the input file (same directory, same
|
||||
basename, .md extension), containing the meeting title/date if found and
|
||||
the speaker-by-speaker transcript with timestamps.
|
||||
"""
|
||||
import email
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from bs4 import BeautifulSoup
|
||||
except ImportError:
|
||||
sys.exit(
|
||||
"Missing dependency 'beautifulsoup4'. Install it with:\n"
|
||||
" pip3 install beautifulsoup4"
|
||||
)
|
||||
|
||||
MONTHS = "Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec"
|
||||
DATE_RE = re.compile(rf"({MONTHS})[a-z]* \d{{1,2}},? \d{{4}}(?:, \d{{1,2}}:\d{{2}} ?[AP]M)?")
|
||||
|
||||
# Fireflies virtualizes the transcript list - if the user didn't scroll
|
||||
# through the whole thing before saving the page, only the visible portion
|
||||
# ends up in the DOM/mhtml, leaving silent gaps. A gap this long between
|
||||
# consecutive timestamps is a strong signal that's what happened.
|
||||
GAP_WARNING_SECONDS = 90
|
||||
|
||||
|
||||
def parse_timestamp(ts):
|
||||
if not ts:
|
||||
return None
|
||||
parts = ts.split(":")
|
||||
try:
|
||||
parts = [int(p) for p in parts]
|
||||
except ValueError:
|
||||
return None
|
||||
seconds = 0
|
||||
for p in parts:
|
||||
seconds = seconds * 60 + p
|
||||
return seconds
|
||||
|
||||
|
||||
def find_large_gaps(lines):
|
||||
gaps = []
|
||||
prev_ts = None
|
||||
prev_idx = None
|
||||
for idx, (ts, _, _) in enumerate(lines):
|
||||
secs = parse_timestamp(ts)
|
||||
if secs is None:
|
||||
continue
|
||||
if prev_ts is not None and secs - prev_ts >= GAP_WARNING_SECONDS:
|
||||
gaps.append((prev_idx, idx, secs - prev_ts))
|
||||
prev_ts, prev_idx = secs, idx
|
||||
return gaps
|
||||
|
||||
|
||||
def load_html_parts(mhtml_path):
|
||||
with open(mhtml_path, "rb") as f:
|
||||
msg = email.message_from_binary_file(f)
|
||||
for part in msg.walk():
|
||||
if part.get_content_type() != "text/html":
|
||||
continue
|
||||
payload = part.get_payload(decode=True)
|
||||
if not payload:
|
||||
continue
|
||||
charset = part.get_content_charset() or "utf-8"
|
||||
yield payload.decode(charset, errors="replace")
|
||||
|
||||
|
||||
def find_transcript_container(html):
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
container = soup.find(id=lambda i: i and i.endswith("content-transcript"))
|
||||
if container is not None:
|
||||
return soup, container
|
||||
# Fallback: some captures use the ScrollArea class directly without the
|
||||
# radix id being present in this particular MIME part.
|
||||
candidates = soup.find_all(
|
||||
class_=lambda c: c and any("ScrollArea-styled__Root" in x for x in (c if isinstance(c, list) else [c]))
|
||||
)
|
||||
if candidates:
|
||||
# The transcript panel is reliably the largest ScrollArea on the page.
|
||||
best = max(candidates, key=lambda el: len(el.get_text()))
|
||||
if len(best.get_text(strip=True)) > 0:
|
||||
return soup, best
|
||||
return soup, None
|
||||
|
||||
|
||||
def extract_paragraphs(container):
|
||||
paragraphs = container.find_all(
|
||||
"div", id=lambda i: i and i.startswith("transcript-paragraph-")
|
||||
)
|
||||
lines = []
|
||||
last_name = None
|
||||
for p in paragraphs:
|
||||
name_span = p.find("span", class_="name")
|
||||
name = name_span.get_text(strip=True) if name_span else (last_name or "Unknown speaker")
|
||||
last_name = name
|
||||
ts_span = p.find("span", attrs={"text-decoration": "underline"})
|
||||
ts = ts_span.get_text(strip=True) if ts_span else ""
|
||||
content_div = p.find("div", class_=lambda c: c and "ContentPost-styled__Content" in c)
|
||||
text = content_div.get_text(" ", strip=True) if content_div else p.get_text(" ", strip=True)
|
||||
if not text:
|
||||
continue
|
||||
lines.append((ts, name, text))
|
||||
return lines
|
||||
|
||||
|
||||
def find_inactive_transcript_tab(html_parts):
|
||||
"""Detect the common failure case: the page was saved with a different
|
||||
tab (usually Notes) active, so Fireflies never mounted the transcript
|
||||
panel into the DOM at all - there's nothing to extract, not a selector
|
||||
mismatch."""
|
||||
for html in html_parts:
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
tab = soup.find(
|
||||
attrs={"role": "tab"},
|
||||
id=lambda i: i and i.endswith("trigger-transcript"),
|
||||
)
|
||||
if tab is not None:
|
||||
active_tab = soup.find(attrs={"role": "tab", "data-state": "active"})
|
||||
active_name = active_tab.get_text(strip=True) if active_tab else "another tab"
|
||||
return tab.get("data-state") != "active", active_name
|
||||
return False, None
|
||||
|
||||
|
||||
def extract_title_and_date(html_parts):
|
||||
title = None
|
||||
date = None
|
||||
for html in html_parts:
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
if title is None and soup.title and soup.title.get_text(strip=True):
|
||||
title = soup.title.get_text(strip=True)
|
||||
if date is None:
|
||||
m = DATE_RE.search(soup.get_text(" ", strip=True))
|
||||
if m:
|
||||
date = m.group(0)
|
||||
if title and date:
|
||||
break
|
||||
return title, date
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 2:
|
||||
sys.exit(f"Usage: {sys.argv[0]} <path/to/page.mhtml>")
|
||||
|
||||
src = Path(sys.argv[1]).expanduser()
|
||||
if not src.is_file():
|
||||
sys.exit(f"File not found: {src}")
|
||||
|
||||
html_parts = list(load_html_parts(src))
|
||||
if not html_parts:
|
||||
sys.exit("No text/html part found in this .mhtml file - is it a valid MIME HTML capture?")
|
||||
|
||||
lines = []
|
||||
for html in html_parts:
|
||||
_, container = find_transcript_container(html)
|
||||
if container is None:
|
||||
continue
|
||||
lines = extract_paragraphs(container)
|
||||
if lines:
|
||||
break
|
||||
|
||||
if not lines:
|
||||
tab_inactive, active_name = find_inactive_transcript_tab(html_parts)
|
||||
if tab_inactive:
|
||||
sys.exit(
|
||||
f"No transcript found - this page was saved with the "
|
||||
f"'{active_name}' tab open, not 'Transcript'. Fireflies only "
|
||||
f"renders the active tab's content into the page, so the "
|
||||
f"transcript panel isn't in this file at all. Reopen the "
|
||||
f"meeting, click the 'Transcript' tab, wait for it to load "
|
||||
f"and scroll to the end (so the full transcript renders), "
|
||||
f"then re-save the page and try again."
|
||||
)
|
||||
sys.exit(
|
||||
"Could not find a transcript panel in this file. This script targets "
|
||||
"Fireflies.ai-style pages (a ScrollArea containing "
|
||||
"'#*-content-transcript'). If the page structure differs, the "
|
||||
"selectors in extract_transcript.py need updating."
|
||||
)
|
||||
|
||||
title, date = extract_title_and_date(html_parts)
|
||||
speakers = sorted(set(name for _, name, _ in lines))
|
||||
|
||||
out_path = src.with_suffix(".md")
|
||||
|
||||
body = []
|
||||
body.append(f"# {title or src.stem}")
|
||||
body.append("")
|
||||
if date:
|
||||
body.append(f"- **Date:** {date}")
|
||||
body.append(f"- **Attendees (speakers detected):** {', '.join(speakers)}")
|
||||
body.append(f"- **Source:** `{src.name}`")
|
||||
body.append("")
|
||||
body.append("## Transcript")
|
||||
body.append("")
|
||||
# Blank line between entries (not a single "\n") so each timestamp
|
||||
# starts its own paragraph in Markdown preview - a lone newline is a
|
||||
# soft break that most renderers collapse into one running paragraph.
|
||||
entries = []
|
||||
for ts, name, text in lines:
|
||||
prefix = f"[{ts}] " if ts else ""
|
||||
entries.append(f"{prefix}{name}: {text}")
|
||||
body.append("\n\n".join(entries))
|
||||
|
||||
out_path.write_text("\n".join(body) + "\n", encoding="utf-8")
|
||||
print(f"Wrote {len(lines)} transcript lines to {out_path}")
|
||||
|
||||
gaps = find_large_gaps(lines)
|
||||
if gaps:
|
||||
print(
|
||||
f"WARNING: {len(gaps)} gap(s) of {GAP_WARNING_SECONDS}s or more "
|
||||
f"between consecutive lines - Fireflies virtualizes the transcript "
|
||||
f"list, so this usually means the page was saved before scrolling "
|
||||
f"through the whole transcript, and content in between is simply "
|
||||
f"missing (not silence). Re-open the meeting, scroll the "
|
||||
f"Transcript tab all the way to the end first, then re-save and "
|
||||
f"re-run this script:",
|
||||
file=sys.stderr,
|
||||
)
|
||||
for start_idx, end_idx, gap_secs in gaps:
|
||||
start_ts = lines[start_idx][0]
|
||||
end_ts = lines[end_idx][0]
|
||||
print(f" - {start_ts} -> {end_ts} ({gap_secs}s gap)", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
196
.agents/skills/fireflies-relanguage/SKILL.md
Normal file
196
.agents/skills/fireflies-relanguage/SKILL.md
Normal file
|
|
@ -0,0 +1,196 @@
|
|||
---
|
||||
name: fireflies-relanguage
|
||||
description: Fix a wrong-language Fireflies transcript on a meeting you don't own (can't hit Reprocess yourself) by re-uploading its audio under the correct language via the Fireflies API, recovering real speaker names into the result with transcript-speaker-fill, then deleting the duplicate meeting. Use when the user has a link to a Fireflies recording they aren't the host/owner of, the transcript is in the wrong language, and they want a corrected transcript without waiting for the owner to act or leaving a duplicate meeting behind. Trigger phrases: "I'm not the owner of this meeting", "can't reprocess this transcript", "fix this Fireflies transcript without the owner", "re-transcribe a shared meeting".
|
||||
---
|
||||
|
||||
# Fireflies relanguage skill
|
||||
|
||||
## Purpose
|
||||
|
||||
Fireflies' own "Update Language + Reprocess" fix only works for the meeting's
|
||||
host/owner (see `guide.fireflies.ai` docs). If you were just sent a link to
|
||||
someone else's recording and it came out in the wrong language, you have no
|
||||
UI path to fix it and can't wait for the owner. This skill routes around
|
||||
that: pull the meeting's audio via the API, re-upload it as a **new** meeting
|
||||
under your own account with the correct language forced, recover the
|
||||
original's real speaker names into the new, correctly-transcribed text using
|
||||
the [[transcript-speaker-fill]](../transcript-speaker-fill/SKILL.md) skill,
|
||||
then delete the duplicate meeting the re-upload created.
|
||||
|
||||
## Before you start: real constraints, not hypothetical ones
|
||||
|
||||
**This may simply not be possible on your account.** `audio_url`/`video_url`
|
||||
on a Fireflies transcript require a Pro+ plan, and it is undocumented whether
|
||||
they populate at all for a meeting merely *shared* with you rather than
|
||||
owned by you. [Michał's own wiki notes](/wiki/entities/fireflies-transcript-handling.md)
|
||||
record hitting exactly this wall before (no Pro seat → transcript download
|
||||
tooling broke). Step 1 below (`fetch`) is also the diagnostic: if it prints
|
||||
the "audio_url is null" warning, stop here — this workflow cannot proceed
|
||||
without one of the workarounds it suggests (a Pro+ seat's API key, the actual
|
||||
owner reprocessing it, or you manually downloading the audio via the web UI
|
||||
and hosting it at some public HTTPS URL yourself).
|
||||
|
||||
**Every re-upload consumes real transcription quota/minutes** on the account
|
||||
whose API key you use, same as any other Fireflies upload — this isn't free
|
||||
just because it's automated. **Deleting a transcript is irreversible.** Don't
|
||||
run this on a meeting you actually might need to keep two copies of, and
|
||||
don't skip the confirmation gates below.
|
||||
|
||||
Also worth knowing up front: the Fireflies API has thin rate limits (Free:
|
||||
50 requests/day, Pro: 500/day, Business/Enterprise: 60/min) and
|
||||
`deleteTranscript` specifically is capped at 10/min. This workflow only
|
||||
needs a handful of calls plus some polling, but don't loop the `wait` step
|
||||
aggressively.
|
||||
|
||||
## Requirements
|
||||
|
||||
- `FIREFLIES_API_KEY` set in the environment. Get one from the Fireflies web
|
||||
app under **Settings > API**. The script exits with this exact instruction
|
||||
if it's missing — don't hardcode the key anywhere.
|
||||
- The `transcript-speaker-fill` skill installed alongside this one (it is,
|
||||
in this repo) — this skill hands off the actual name-recovery merge to it
|
||||
rather than duplicating that logic.
|
||||
|
||||
## Workflow
|
||||
|
||||
Resolve `<skill-dir>` to this skill's own directory throughout.
|
||||
|
||||
### 1. Fetch the original (broken-language) transcript
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fireflies_client.py" fetch --ref "<link-or-id-the-user-gave-you>" --out original.md
|
||||
```
|
||||
|
||||
Accepts a bare transcript id or a full share link like
|
||||
`https://app.fireflies.ai/view/Some-Title::abcDEF123`. Writes `original.md`
|
||||
(real speaker names, garbled text — the shape `transcript-speaker-fill`
|
||||
expects for its "broken" input) and `original.md.meta.json` (id, title,
|
||||
`audio_url`, etc., needed by later steps).
|
||||
|
||||
**Check the printed output for the audio_url warning before continuing.** If
|
||||
it's null, stop and follow the guidance it prints instead of proceeding to
|
||||
step 2 — do not attempt step 2 anyway "just to see."
|
||||
|
||||
### 2. Confirm the target language with the user
|
||||
|
||||
Don't guess the correct language from context alone unless it's unambiguous
|
||||
(e.g. the user already told you). Ask if unclear. Use the language code
|
||||
Fireflies expects — check
|
||||
[Learn about Fireflies supported languages](https://guide.fireflies.ai/articles/2973706448-learn-about-fireflies-supported-languages)
|
||||
for the exact code if you're not sure it matches (e.g. `en`, `pl`, `es`).
|
||||
|
||||
### 3. Re-upload the audio under the correct language
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fireflies_client.py" upload --meta original.md.meta.json --language <code>
|
||||
```
|
||||
|
||||
This calls `uploadAudio` with `custom_language` set, tagging the new
|
||||
meeting's title with `[relang:<original-id>]` so it can be found
|
||||
unambiguously afterward (override with `--title` if you want a cleaner
|
||||
name, but then pass that exact same string to step 4). Note the exact title
|
||||
printed — you need it verbatim for the next step.
|
||||
|
||||
### 4. Wait for the re-upload to finish processing
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fireflies_client.py" wait --title "<exact title from step 3>"
|
||||
```
|
||||
|
||||
Polls every 30s (default) up to 15 minutes (default) for a transcript with
|
||||
that exact title to appear with content. Long recordings can take longer —
|
||||
if it times out, just re-run `wait` again rather than assuming failure.
|
||||
Prints the new transcript's `id` once ready.
|
||||
|
||||
### 5. Fetch the new (correct-language) transcript
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fireflies_client.py" fetch --ref "<id from step 4>" --out new.md
|
||||
```
|
||||
|
||||
This one should come back with correct text but generic `Speaker 1`,
|
||||
`Speaker 2`, ... labels (no calendar/roster context on a bare re-upload) —
|
||||
exactly the gap `transcript-speaker-fill` closes.
|
||||
|
||||
### 6. Recover real speaker names
|
||||
|
||||
Hand off to the `transcript-speaker-fill` skill exactly per its own
|
||||
SKILL.md, using `original.md` and `new.md` as the two inputs — dry run
|
||||
first, present the resolution report, get the user's confirmation/manual
|
||||
overrides, then `--apply`. Do not skip its confirmation gate just because
|
||||
you're mid-pipeline; it's exactly as load-bearing here as when invoked
|
||||
standalone.
|
||||
|
||||
Also pass `--recording-url "https://app.fireflies.ai/view/<id from step 4>"`
|
||||
on every `fill_speakers.py` call in this handoff (both the dry-run and the
|
||||
`--apply` run) — the bare-id form of the URL works without needing to
|
||||
slug-encode the title, and gives the user a direct link to a recording they
|
||||
definitely have full access to (they own the re-uploaded meeting) right
|
||||
next to every label's example lines, for the spot-check
|
||||
`transcript-speaker-fill` asks them to do before confirming. Use the new
|
||||
transcript's id, not the original's — do this *before* step 8's cleanup,
|
||||
while that meeting (and its recording) still exists.
|
||||
|
||||
### 7. Ask where to save the corrected transcript
|
||||
|
||||
Before touching the duplicate meeting, ask the user where the final
|
||||
`--apply`'d output (from step 6) should end up — don't default to leaving
|
||||
it in a scratch/temp location without asking. Common answers: a specific
|
||||
path they name, this project's own `tmp/` (if this skill is being run from
|
||||
inside a Cascade Knowledge Base repo like this one — gitignored, agent-
|
||||
managed), or `raw/inbox/` if they want it ingested into a wiki afterward.
|
||||
Move (don't copy) the file there once they've told you.
|
||||
|
||||
### 8. Delete the duplicate meeting — only after explicit confirmation
|
||||
|
||||
Once the user has confirmed the merged transcript (from step 6) looks right
|
||||
and it's been saved wherever they wanted (step 7), delete the meeting the
|
||||
re-upload created (**not** the original — you don't own that one anyway,
|
||||
and couldn't delete it if you tried):
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fireflies_client.py" delete --id <id from step 4> --yes
|
||||
```
|
||||
|
||||
The script refuses to run without `--yes`. Never pass `--yes` without the
|
||||
user having explicitly confirmed they're ready — this is irreversible and
|
||||
removes a real meeting from their Fireflies account. Show them the printed
|
||||
`{id, title, date, duration}` of what was deleted as final confirmation.
|
||||
|
||||
## Failure modes and what they mean
|
||||
|
||||
- **`fetch` on the original prints the audio_url warning** — see "Before you
|
||||
start" above. This is the expected failure mode when the API key's
|
||||
account isn't Pro+, or when a merely-shared meeting doesn't expose audio
|
||||
via the API. Not a bug to work around silently.
|
||||
- **`upload` mutation succeeds (`success: true`) but `wait` never finds it**
|
||||
— the title match is exact-string, so a `--title` override that doesn't
|
||||
exactly match what you pass to `wait` will never resolve; double check
|
||||
you used the identical string in both commands.
|
||||
- **`wait` times out** — normal for long recordings. Re-run it; don't
|
||||
assume the upload failed.
|
||||
- **Rate limit errors (`too_many_requests`)** — the error message includes
|
||||
`retryAfter`; wait that long before retrying, especially on a Free-tier
|
||||
key (50 requests/day total).
|
||||
- **`delete` fails with `require_elevated_privilege`** — you're trying to
|
||||
delete a transcript you don't own (e.g. you accidentally passed the
|
||||
*original* id instead of the new one from step 4). Only the re-uploaded
|
||||
meeting is yours to delete.
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **Multiple speakers with a name-recovery gap** (new transcript has more
|
||||
distinct `Speaker N` labels than the original has real names) — this is
|
||||
`transcript-speaker-fill`'s "roster gap" warning, not something this
|
||||
skill's own steps can fix; it means someone's voice wasn't distinctly
|
||||
captured with a real name in the original either.
|
||||
- **Original meeting has `video_url` but not `audio_url`** — `upload` only
|
||||
accepts `--audio-url`; there's no video re-upload path here. If you truly
|
||||
only have video access, extract audio from it yourself first and host
|
||||
that as a public URL, then pass it via `--audio-url`.
|
||||
- **The user wants the corrected transcript ingested into this repo's
|
||||
wiki**, not just saved as a file — step 7 already asks where to save it;
|
||||
if the answer is "ingest it," save it to `raw/inbox/` there, then run the
|
||||
normal "Ingest" workflow from the root `CLAUDE.md` on it after step 8's
|
||||
cleanup (the duplicate meeting is Fireflies-side bookkeeping, unrelated
|
||||
to whether the wiki ingest has happened yet).
|
||||
337
.agents/skills/fireflies-relanguage/scripts/fireflies_client.py
Normal file
337
.agents/skills/fireflies-relanguage/scripts/fireflies_client.py
Normal file
|
|
@ -0,0 +1,337 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Minimal Fireflies.ai GraphQL client for the fireflies-relanguage skill.
|
||||
|
||||
Handles the four network steps of the workflow: pull a transcript you can
|
||||
view (owned or shared) into the Markdown shape transcript-speaker-fill
|
||||
expects, kick off a re-upload of its audio under the correct language,
|
||||
poll until that re-upload shows up as a finished transcript, and delete
|
||||
the resulting duplicate meeting once you're done with it.
|
||||
|
||||
No third-party dependencies - stdlib only (urllib), so this runs with a
|
||||
bare `python3` on any machine that has the transcript-speaker-fill skill
|
||||
installed.
|
||||
|
||||
Requires FIREFLIES_API_KEY in the environment (Settings > API in the
|
||||
Fireflies web app to generate one).
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
API_URL = "https://api.fireflies.ai/graphql"
|
||||
GENERIC_RE = re.compile(r'^Speaker\s+\d+$')
|
||||
|
||||
|
||||
def parse_transcript_ref(ref):
|
||||
"""Accept a bare id, or a shared link like
|
||||
https://app.fireflies.ai/view/Some-Title::abcDEF123?channelSource=mine-shared
|
||||
(query string / fragment and the ::title part are both optional) and
|
||||
return just the transcript id."""
|
||||
ref = ref.strip()
|
||||
if '://' in ref:
|
||||
ref = ref.split('?', 1)[0].split('#', 1)[0]
|
||||
ref = ref.rstrip('/').rsplit('/', 1)[-1]
|
||||
if '::' in ref:
|
||||
ref = ref.rsplit('::', 1)[-1]
|
||||
return ref
|
||||
|
||||
|
||||
def gql(api_key, query, variables=None):
|
||||
body = json.dumps({"query": query, "variables": variables or {}}).encode('utf-8')
|
||||
req = urllib.request.Request(
|
||||
API_URL,
|
||||
data=body,
|
||||
method='POST',
|
||||
headers={
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': f'Bearer {api_key}',
|
||||
},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
payload = json.loads(resp.read().decode('utf-8'))
|
||||
except urllib.error.HTTPError as e:
|
||||
raw = e.read().decode('utf-8', errors='replace')
|
||||
sys.exit(f"HTTP {e.code} from Fireflies API: {raw}")
|
||||
|
||||
if payload.get('errors'):
|
||||
msgs = []
|
||||
for err in payload['errors']:
|
||||
ext = err.get('extensions') or {}
|
||||
code = ext.get('code', '')
|
||||
msg = err.get('message', '')
|
||||
extra = f" (code={code}" + (f", retryAfter={ext['retryAfter']}" if 'retryAfter' in ext else '') + ')' if code else ''
|
||||
msgs.append(f"{msg}{extra}")
|
||||
sys.exit("Fireflies API returned error(s): " + " | ".join(msgs))
|
||||
return payload.get('data') or {}
|
||||
|
||||
|
||||
def require_api_key():
|
||||
key = os.environ.get('FIREFLIES_API_KEY')
|
||||
if not key:
|
||||
sys.exit(
|
||||
"FIREFLIES_API_KEY is not set. Generate a key in the Fireflies web "
|
||||
"app under Settings > API, then export it, e.g.:\n"
|
||||
" export FIREFLIES_API_KEY=your-key-here"
|
||||
)
|
||||
return key
|
||||
|
||||
|
||||
def format_ts(seconds):
|
||||
seconds = float(seconds)
|
||||
total = int(round(seconds))
|
||||
h, rem = divmod(total, 3600)
|
||||
m, s = divmod(rem, 60)
|
||||
if h:
|
||||
return f"{h}:{m:02d}:{s:02d}"
|
||||
return f"{m}:{s:02d}"
|
||||
|
||||
|
||||
TRANSCRIPT_FIELDS = """
|
||||
id
|
||||
title
|
||||
dateString
|
||||
date
|
||||
duration
|
||||
audio_url
|
||||
video_url
|
||||
sentences {
|
||||
speaker_name
|
||||
speaker_id
|
||||
start_time
|
||||
end_time
|
||||
text
|
||||
raw_text
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def cmd_fetch(args):
|
||||
api_key = require_api_key()
|
||||
tid = parse_transcript_ref(args.ref)
|
||||
data = gql(
|
||||
api_key,
|
||||
f"query Transcript($id: String!) {{ transcript(id: $id) {{ {TRANSCRIPT_FIELDS} }} }}",
|
||||
{"id": tid},
|
||||
)
|
||||
t = data.get('transcript')
|
||||
if not t:
|
||||
sys.exit(
|
||||
f"No transcript returned for id '{tid}'. Either the id is wrong, or this "
|
||||
f"API key's account doesn't have access to it (not shared with you, or "
|
||||
f"the workspace/plan doesn't expose it via API)."
|
||||
)
|
||||
|
||||
sentences = t.get('sentences') or []
|
||||
if not sentences:
|
||||
sys.exit(
|
||||
f"Transcript '{t.get('title')}' ({tid}) has no sentences yet - it may "
|
||||
f"still be processing, or your account tier doesn't return transcript "
|
||||
f"content via the API for this meeting."
|
||||
)
|
||||
|
||||
out_path = Path(args.out)
|
||||
lines = []
|
||||
for s in sentences:
|
||||
speaker = s.get('speaker_name') or f"Speaker {s.get('speaker_id', '?')}"
|
||||
text = s.get('text') or s.get('raw_text') or ''
|
||||
ts = format_ts(s.get('start_time', 0))
|
||||
lines.append(f"**{speaker}** *[{ts}]*: {text}")
|
||||
out_path.write_text("\n".join(lines) + "\n", encoding='utf-8')
|
||||
|
||||
meta = {
|
||||
'id': t['id'],
|
||||
'title': t.get('title'),
|
||||
'dateString': t.get('dateString'),
|
||||
'date': t.get('date'),
|
||||
'duration': t.get('duration'),
|
||||
'audio_url': t.get('audio_url'),
|
||||
'video_url': t.get('video_url'),
|
||||
'sentence_count': len(sentences),
|
||||
}
|
||||
meta_path = out_path.with_suffix(out_path.suffix + '.meta.json')
|
||||
meta_path.write_text(json.dumps(meta, indent=2), encoding='utf-8')
|
||||
|
||||
print(f"Wrote {len(lines)} lines to {out_path}")
|
||||
print(f"Wrote metadata to {meta_path}")
|
||||
print(f"Title: {meta['title']}")
|
||||
print(f"audio_url present: {bool(meta['audio_url'])}")
|
||||
print(f"video_url present: {bool(meta['video_url'])}")
|
||||
if not meta['audio_url']:
|
||||
print(
|
||||
"\nWARNING: audio_url is null. This is the expected failure mode when the "
|
||||
"querying account isn't on a Fireflies Pro+ seat, or when audio access "
|
||||
"isn't granted to a merely-shared (non-owned) meeting. Re-uploading this "
|
||||
"meeting's audio via the API is NOT possible until this is resolved - "
|
||||
"either use an API key belonging to a Pro+ seat, ask the meeting owner to "
|
||||
"reprocess it directly, or manually download the audio via the Fireflies "
|
||||
"web UI (if the share settings allow it) and host it at a public HTTPS URL "
|
||||
"yourself before using the `upload` command's --audio-url override.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
|
||||
def cmd_upload(args):
|
||||
api_key = require_api_key()
|
||||
meta = json.loads(Path(args.meta).read_text(encoding='utf-8'))
|
||||
|
||||
audio_url = args.audio_url or meta.get('audio_url')
|
||||
if not audio_url:
|
||||
sys.exit(
|
||||
"No audio_url available (neither in the meta file nor via --audio-url). "
|
||||
"See the WARNING printed by the `fetch` command for why, and how to work "
|
||||
"around it."
|
||||
)
|
||||
|
||||
original_id = meta['id']
|
||||
title = args.title or f"{meta.get('title', 'Meeting')} [relang-{original_id}]"
|
||||
|
||||
variables = {
|
||||
"input": {
|
||||
"url": audio_url,
|
||||
"title": title,
|
||||
"custom_language": args.language,
|
||||
"client_reference_id": original_id,
|
||||
}
|
||||
}
|
||||
if args.bypass_size_check:
|
||||
variables["input"]["bypass_size_check"] = True
|
||||
|
||||
data = gql(
|
||||
api_key,
|
||||
"""
|
||||
mutation UploadAudio($input: AudioUploadInput!) {
|
||||
uploadAudio(input: $input) {
|
||||
success
|
||||
title
|
||||
message
|
||||
}
|
||||
}
|
||||
""",
|
||||
variables,
|
||||
)
|
||||
result = data.get('uploadAudio') or {}
|
||||
echoed_title = result.get('title') or title
|
||||
print(json.dumps({
|
||||
"submitted_title": title,
|
||||
"language": args.language,
|
||||
"source_transcript_id": original_id,
|
||||
"api_response": result,
|
||||
}, indent=2))
|
||||
if not result.get('success'):
|
||||
sys.exit("uploadAudio did not report success - check the message above.")
|
||||
if echoed_title != title:
|
||||
print(
|
||||
f"\nNOTE: Fireflies echoed back a different title than submitted "
|
||||
f"(likely stripped/altered some character) - use the ECHOED one below "
|
||||
f"for `wait`, not what you submitted."
|
||||
)
|
||||
print(
|
||||
f"\nQueued. Use the `wait` command with --title '{echoed_title}' to find the "
|
||||
f"new transcript once processing finishes (this can take several minutes for "
|
||||
f"a long recording)."
|
||||
)
|
||||
|
||||
|
||||
def cmd_wait(args):
|
||||
api_key = require_api_key()
|
||||
deadline = time.time() + args.max_wait
|
||||
attempt = 0
|
||||
while True:
|
||||
attempt += 1
|
||||
data = gql(
|
||||
api_key,
|
||||
"""
|
||||
query Transcripts($limit: Int) {
|
||||
transcripts(mine: true, limit: $limit) {
|
||||
id
|
||||
title
|
||||
dateString
|
||||
sentences { speaker_name }
|
||||
}
|
||||
}
|
||||
""",
|
||||
{"limit": args.list_limit},
|
||||
)
|
||||
candidates = [t for t in (data.get('transcripts') or []) if t.get('title') == args.title]
|
||||
ready = [t for t in candidates if t.get('sentences')]
|
||||
if ready:
|
||||
t = ready[0]
|
||||
print(json.dumps({"id": t['id'], "title": t['title'], "dateString": t.get('dateString'), "ready": True}, indent=2))
|
||||
return
|
||||
if candidates:
|
||||
print(f"[attempt {attempt}] Found the meeting but it's still processing (no sentences yet)...", file=sys.stderr)
|
||||
else:
|
||||
print(f"[attempt {attempt}] Not found yet...", file=sys.stderr)
|
||||
if time.time() >= deadline:
|
||||
sys.exit(
|
||||
f"Gave up after {args.max_wait}s without finding a ready transcript "
|
||||
f"titled '{args.title}'. Long recordings can take longer than that to "
|
||||
f"process - re-run `wait` again with a fresh --max-wait, or check the "
|
||||
f"Fireflies web UI directly for a meeting with that title."
|
||||
)
|
||||
time.sleep(args.interval)
|
||||
|
||||
|
||||
def cmd_delete(args):
|
||||
if not args.yes:
|
||||
sys.exit("Refusing to delete without --yes (this is irreversible).")
|
||||
api_key = require_api_key()
|
||||
data = gql(
|
||||
api_key,
|
||||
"""
|
||||
mutation DeleteTranscript($id: String!) {
|
||||
deleteTranscript(id: $id) {
|
||||
id
|
||||
title
|
||||
date
|
||||
duration
|
||||
}
|
||||
}
|
||||
""",
|
||||
{"id": args.id},
|
||||
)
|
||||
print(json.dumps(data.get('deleteTranscript') or {}, indent=2))
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
sub = ap.add_subparsers(dest='command', required=True)
|
||||
|
||||
p = sub.add_parser('fetch', help="Fetch a transcript (owned or shared) into transcript-speaker-fill-compatible Markdown + a metadata sidecar.")
|
||||
p.add_argument('--ref', required=True, help='Transcript id, or a Fireflies share URL/link')
|
||||
p.add_argument('--out', required=True, help='Output .md path')
|
||||
p.set_defaults(func=cmd_fetch)
|
||||
|
||||
p = sub.add_parser('upload', help="Re-upload a transcript's audio under a different language via uploadAudio.")
|
||||
p.add_argument('--meta', required=True, help="Path to the .meta.json produced by `fetch` for the ORIGINAL (broken-language) transcript")
|
||||
p.add_argument('--language', required=True, help='Target language code, e.g. "en", "pl", "es"')
|
||||
p.add_argument('--title', help='Override the title used for the new upload (default: original title + a [relang-<id>] tag)')
|
||||
p.add_argument('--audio-url', help='Override the audio URL instead of using the one from --meta (e.g. a self-hosted fallback URL)')
|
||||
p.add_argument('--bypass-size-check', action='store_true')
|
||||
p.set_defaults(func=cmd_upload)
|
||||
|
||||
p = sub.add_parser('wait', help='Poll until the re-uploaded transcript shows up as fully processed.')
|
||||
p.add_argument('--title', required=True, help='Exact title echoed back by `upload` (not necessarily what you submitted - Fireflies can alter it, e.g. stripping colons)')
|
||||
p.add_argument('--interval', type=int, default=30, help='Seconds between polls (default 30)')
|
||||
p.add_argument('--max-wait', type=int, default=900, help='Give up after this many seconds (default 900 = 15 min)')
|
||||
p.add_argument('--list-limit', type=int, default=20, help="How many of your most recent transcripts to scan each poll for an exact title match (default 20; server 'keyword' search was found unreliable against bracket-tagged titles, so this lists recent transcripts client-side instead of filtering server-side)")
|
||||
p.set_defaults(func=cmd_wait)
|
||||
|
||||
p = sub.add_parser('delete', help='Delete a transcript by id (irreversible).')
|
||||
p.add_argument('--id', required=True)
|
||||
p.add_argument('--yes', action='store_true', help='Required confirmation flag')
|
||||
p.set_defaults(func=cmd_delete)
|
||||
|
||||
args = ap.parse_args()
|
||||
args.func(args)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
177
.agents/skills/project-summary/SKILL.md
Normal file
177
.agents/skills/project-summary/SKILL.md
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
---
|
||||
name: project-summary
|
||||
description: Generate or refresh a one-to-two-page project overview at the repo root, synthesized entirely from the current wiki/ contents — overview paragraph, project state, actions and their status, risks, and assumptions. Use when the user asks for "a project summary", "project overview", "give me the state of the project", "where do things stand", "summarize the project", or wants a quick top-level snapshot without reading the whole wiki.
|
||||
---
|
||||
|
||||
# Project summary skill
|
||||
|
||||
## Purpose
|
||||
|
||||
Produce a short, standalone snapshot of the project at repo root - something
|
||||
a busy stakeholder can read in two minutes without opening `wiki/` at all.
|
||||
Everything in it must trace back to what the wiki actually says; this is a
|
||||
synthesis of existing knowledge, not a place to add new claims or guesses.
|
||||
|
||||
This is a one-way, on-demand export - `wiki/` stays the authoritative
|
||||
source; the generated overview file is always a derived, refreshable
|
||||
snapshot. Re-running the skill regenerates it from whatever the wiki says
|
||||
*now*; it does not append to the previous version.
|
||||
|
||||
## Trigger phrases
|
||||
|
||||
Use this skill when the user says things like:
|
||||
- "give me a project summary" / "project overview"
|
||||
- "where do things stand" / "what's the state of the project"
|
||||
- "summarize the project" / "one-pager" / "two-pager"
|
||||
- "catch me up on the project"
|
||||
|
||||
## How to run this skill
|
||||
|
||||
### Step 1 - Identify the project
|
||||
|
||||
Read `wiki/index.md` and `wiki/entities/index.md` first (small, cheap,
|
||||
gives the lay of the land). Then find every entity page whose frontmatter
|
||||
has `type: project`.
|
||||
|
||||
- **Exactly one** - that's the subject, proceed.
|
||||
- **None found** - tell the user no `type: project` entity exists yet in
|
||||
the wiki, and ask whether to point at a specific entity page instead or
|
||||
run Ingest first. Do not fabricate a project summary from thin air.
|
||||
- **More than one** - list them (title + `tldr`) and ask which one to
|
||||
summarize, or whether they want a separate overview file for each.
|
||||
|
||||
### Step 2 - Gather context via the graph, not just one page
|
||||
|
||||
Read the chosen project entity page in full. Then read
|
||||
`wiki/graph/edges.json` and pull every edge where this entity is the
|
||||
`source` or `target` - these are its direct dependencies. Read each of
|
||||
those pages in full too. If a dependency page itself has further edges
|
||||
that look load-bearing (e.g. another concept it depends on), follow one
|
||||
more hop - but stop there. This is a snapshot, not a full graph traversal;
|
||||
two hops is enough to be accurate without pulling in the entire wiki.
|
||||
|
||||
Also check the single most recent `workload/YYYY-MM-DD_summary.md` (by
|
||||
filename date) for anything very recent that may not have made it into the
|
||||
wiki yet. Treat the wiki as authoritative if the two conflict - workload
|
||||
files are a running log, not a maintained source of truth - but note a
|
||||
real discrepancy in Project State rather than silently picking one.
|
||||
|
||||
### Step 3 - Extract and classify
|
||||
|
||||
Work through everything gathered in Step 2 and sort content into five
|
||||
buckets. Do not pad any section to look complete - a short, honest section
|
||||
is better than filler, and it's fine for Risks or Assumptions to be thin or
|
||||
even say "none currently recorded" if that's genuinely the case.
|
||||
|
||||
1. **Overview** - one short paragraph: what the project is, who's involved
|
||||
(client / delivery org / key roles), and what it's for. This is
|
||||
scene-setting, not a list of facts.
|
||||
|
||||
2. **Project State** - the current concrete status: what phase things are
|
||||
in, what's confirmed vs. pending, key dates. Pull directly from
|
||||
"Current Shape" / "Management Summary" / status-like sections on the
|
||||
entity pages gathered.
|
||||
|
||||
3. **Actions & Status** - every concrete to-do, next step, or open item
|
||||
found across the gathered pages (commonly under headings like "Open
|
||||
Points" or "Open items"). Dedupe overlapping items from different pages
|
||||
into one line. Assign a status to each from this fixed vocabulary based
|
||||
on the page's own wording - don't invent a finer-grained scheme:
|
||||
- **Done** - completed, confirmed, signed, resolved
|
||||
- **In progress** - actively being worked, partially complete
|
||||
- **Blocked** - explicitly waiting on something else before it can move
|
||||
- **Open** - identified but not yet started or scheduled
|
||||
|
||||
4. **Risks** - things that could hurt the project if they go wrong or stay
|
||||
unresolved. Look for language like "risk", "concern", "gap", "blocked
|
||||
on", "unresolved", "dependent on", "may not stick", "left dependent on."
|
||||
For each: what the risk is, why it matters, and - if the source says so
|
||||
- what would mitigate or resolve it.
|
||||
|
||||
5. **Assumptions** - things the current plan depends on being true but that
|
||||
aren't yet confirmed. Look for language like "expected", "should",
|
||||
"likely", "assuming", "TBC", "to be confirmed", "time-bound", "reconfirm
|
||||
before acting." For each: what's assumed, and what would need to happen
|
||||
to confirm or invalidate it.
|
||||
|
||||
An item can legitimately touch both Risks and Assumptions (an unconfirmed
|
||||
assumption is often also a risk if it turns out false) - when that
|
||||
happens, put it wherever it fits better and don't duplicate it in both.
|
||||
|
||||
If there's more candidate material than fits a one-to-two-pager, prioritize
|
||||
by what the source pages themselves flag as higher-confidence, higher-
|
||||
retention, and most recently updated - this is a snapshot of what matters
|
||||
most right now, not an exhaustive appendix.
|
||||
|
||||
### Step 4 - Write the file
|
||||
|
||||
Target length: one to two pages (roughly 500-900 words total, including
|
||||
bullets - err short over long). Write to `PROJECT-OVERVIEW.md` at the repo
|
||||
root, overwriting it completely if it already exists (this is a refreshable
|
||||
snapshot, never hand-append to a previous version).
|
||||
|
||||
Use this structure:
|
||||
|
||||
```markdown
|
||||
# [Project Name] - Project Overview
|
||||
|
||||
*Auto-generated from `wiki/` - do not hand-edit; regenerating this skill overwrites this file. Last generated: YYYY-MM-DD.*
|
||||
|
||||
**At a glance:** [client] · [delivery org] · [one-line phase/status]
|
||||
|
||||
## Overview
|
||||
|
||||
[the synthesis paragraph]
|
||||
|
||||
## Project State
|
||||
|
||||
[status bullets or short paragraph]
|
||||
|
||||
## Actions & Status
|
||||
|
||||
| Action | Status |
|
||||
|---|---|
|
||||
| ... | Done / In progress / Blocked / Open |
|
||||
|
||||
## Risks
|
||||
|
||||
- **[risk]** - [why it matters] [mitigation if known]
|
||||
|
||||
## Assumptions
|
||||
|
||||
- **[assumption]** - [what would confirm/invalidate it]
|
||||
|
||||
---
|
||||
*Sources: [wiki pages used, as markdown links]*
|
||||
```
|
||||
|
||||
Use plain markdown links to the wiki pages actually used (e.g.
|
||||
`[Grant Thornton FDE engagement](wiki/entities/grant-thornton-fde-engagement.md)`)
|
||||
so a reader can jump to the full detail behind any line. Keep the source
|
||||
list to what was actually used, not every page in the wiki.
|
||||
|
||||
If more than one `type: project` entity was chosen for separate summaries
|
||||
(Step 1), name the files `PROJECT-OVERVIEW-<slug>.md` instead of the plain
|
||||
name, using a short kebab-case slug of the project's title.
|
||||
|
||||
### Step 5 - Report
|
||||
|
||||
Tell the user the file was written (path), give a one-line gist of what it
|
||||
found (e.g. "3 open actions, 2 risks, 1 assumption"), and mention it can be
|
||||
regenerated any time by re-running this skill - it will always reflect
|
||||
whatever the wiki says at that moment, not what it said today.
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **Empty or near-empty project entity page** - write a short, honest
|
||||
Overview and State, and let Actions/Risks/Assumptions be genuinely short
|
||||
rather than inventing content to fill the template.
|
||||
- **Conflicting information between two source pages** - prefer the page
|
||||
with the more recent `last_updated`; if they're the same date, note the
|
||||
conflict briefly in Project State rather than silently picking one.
|
||||
- **`PROJECT-OVERVIEW.md` already exists** - overwrite it; this file is
|
||||
always meant to reflect the current wiki, not accumulate history. If the
|
||||
user wants history, that's what `wiki/log.md` and `workload/` are for.
|
||||
- **Re-running with no wiki changes since the last run** - should produce
|
||||
essentially the same content each time; don't introduce random variation
|
||||
in section content or ordering between runs.
|
||||
302
.agents/skills/transcript-speaker-fill/SKILL.md
Normal file
302
.agents/skills/transcript-speaker-fill/SKILL.md
Normal file
|
|
@ -0,0 +1,302 @@
|
|||
---
|
||||
name: transcript-speaker-fill
|
||||
description: Recover real speaker names in a Fireflies transcript that only has generic "Speaker N" labels, by fuzzy-matching timestamps against a second transcript of the same meeting that has real names but broken/unusable text (commonly caused by wrong-language detection). Use when the user has two Fireflies exports of one meeting - one with correct text but no speaker names, one with real names but garbled text - and wants the names filled into the good transcript. Trigger phrases: "fill in the speakers", "match speakers by timestamp", "recover speaker names", "one transcript is missing speakers and the other has the wrong language".
|
||||
---
|
||||
|
||||
# Transcript speaker fill skill
|
||||
|
||||
## Purpose
|
||||
|
||||
Fireflies occasionally produces two broken outcomes for the same meeting:
|
||||
one export has correct transcript text but generic `Speaker 1`, `Speaker 2`,
|
||||
... labels (diarization worked, but names were never resolved or the
|
||||
Fireflies bot lost the participant roster); another export - often from a
|
||||
retry after fixing the language setting - has real speaker names but
|
||||
garbage text (wrong language was detected, so the words are nonsense, but
|
||||
the underlying speaker diarization and timestamps are still meaningful).
|
||||
|
||||
This skill cross-references the two: it can't read the garbled text, but it
|
||||
can compare *when* each speaker was talking, and use that timing to guess
|
||||
which generic label corresponds to which real name in the good transcript.
|
||||
|
||||
This is entirely mechanical - a bundled Python script does the parsing,
|
||||
offset detection, and voting. Nothing here needs model judgment except
|
||||
interpreting the final report and deciding whether the result is trustworthy
|
||||
enough to use.
|
||||
|
||||
## Important: this is best-effort, not a solved match
|
||||
|
||||
Be direct with the user about this before and after running it:
|
||||
|
||||
- **Some speakers may be structurally unresolvable.** If the target
|
||||
transcript has more distinct unnamed speakers than the broken transcript
|
||||
has distinct real names, some target speakers simply aren't captured
|
||||
with a real name *anywhere* in the broken file (they may have joined
|
||||
late, or their voice wasn't separated out in that broken run). No amount
|
||||
of tuning fixes this - the script detects and reports this gap explicitly,
|
||||
but can still produce a confident-*looking* wrong answer for an affected
|
||||
label, because it has no way to know a name is entirely absent from the
|
||||
candidate pool.
|
||||
- **Timestamps are start-of-utterance markers from two independent
|
||||
diarization runs**, not a shared clock - they can disagree by several
|
||||
seconds even for a genuine match, and the two files may have a constant
|
||||
offset if the bots didn't start recording at exactly the same instant.
|
||||
The script searches for that offset automatically; it can still get it
|
||||
slightly wrong in a noisy recording.
|
||||
- **Always spot-check the result** - especially any label resolved with
|
||||
fewer than ~4-5 votes or under ~65% confidence, and *especially* any
|
||||
label affected by the roster-gap warning. Read a couple of the actual
|
||||
lines attributed to a resolved name and sanity-check against tone/content
|
||||
(a name attributed to a monologue about hairdressers when the person is
|
||||
known to talk mostly about delivery process, say, is a red flag).
|
||||
|
||||
## How to run this skill
|
||||
|
||||
This is a two-pass flow: a dry-run preview first, a write-to-disk second,
|
||||
only after the matches have been confirmed with the user.
|
||||
|
||||
### Pass 1 - preview (dry run, default)
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fill_speakers.py" "<file-1>" "<file-2>"
|
||||
```
|
||||
|
||||
- `<file-1>` / `<file-2>` - the two transcripts, **in either order**. The
|
||||
script auto-detects which one is the broken source (real speaker names,
|
||||
garbled/wrong-language text) and which is the target (correct text,
|
||||
generic `Speaker N` labels) by comparing how much of each file is still
|
||||
labeled with generic `Speaker N` names - it does **not** rely on file
|
||||
size or on which argument came first. It prints which role it assigned
|
||||
to each file near the top of the output - check that this matches what
|
||||
you'd expect from the filenames/content before trusting the rest of the
|
||||
report. If the two files are too similar to tell apart (rare), it exits
|
||||
with an error instead of guessing - open both and check by eye: the
|
||||
broken one reads as nonsense/wrong language but has real names; the
|
||||
target reads correctly but has `Speaker N` labels.
|
||||
|
||||
Resolve `<skill-dir>` to this skill's own directory.
|
||||
|
||||
With no `--apply` flag, the script **only prints the matching report** -
|
||||
merges, roster-gap warning, offset, the per-label resolution table, and
|
||||
any unresolved labels with text snippets. It does not touch disk yet.
|
||||
Present this report to the user (see "After the preview" below) and get
|
||||
their explicit confirmation - or their corrections via `--manual` - before
|
||||
moving to pass 2.
|
||||
|
||||
### Pass 2 - apply (only after confirmation)
|
||||
|
||||
Once the user has confirmed the matches (and supplied any `--manual`
|
||||
overrides for gaps or corrections), re-run the exact same command with
|
||||
`--apply` added:
|
||||
|
||||
```bash
|
||||
python3 "<skill-dir>/scripts/fill_speakers.py" "<file-1>" "<file-2>" --apply [--manual "Speaker N=Real Name" ...]
|
||||
```
|
||||
|
||||
Only this run writes the output file. It never modifies either input
|
||||
file - it writes a new file next to the target, named
|
||||
`<target-stem>-speakers-filled.md` by default (override with `--output
|
||||
<path>`), with every generic label it could confidently resolve (or that
|
||||
was given via `--manual`) replaced by the real name throughout the
|
||||
target - one resolution per label, applied consistently everywhere that
|
||||
label appears, not line-by-line guessing.
|
||||
|
||||
Do not pass `--apply` on the first run, and do not treat pass 1's report
|
||||
as final - it is a proposal for the user to react to, not a completed
|
||||
action.
|
||||
|
||||
Useful tuning flags if the default report looks too conservative or too
|
||||
noisy:
|
||||
- `--tolerance <seconds>` (default 15) - how close two timestamps must be
|
||||
to count as a candidate match.
|
||||
- `--min-votes <n>` (default 2) / `--min-confidence <0-1>` (default 0.5) -
|
||||
how many matches, and what vote share, a label needs before it gets
|
||||
resolved instead of left as `Speaker N`. Raise both for a more
|
||||
conservative (fewer, more trustworthy) result.
|
||||
- `--offset-search <seconds>` (default 90) - how wide a window to search
|
||||
for a constant clock offset between the two recordings.
|
||||
- `--no-normalize` - by default, the broken file's own duplicate-diarization
|
||||
name variants are merged before voting (see next section) - most
|
||||
commonly a trailing digit Fireflies adds when it's unsure two segments
|
||||
are the same voice cluster (e.g. `Robert Drazkowski` and
|
||||
`Robert Drazkowski1` collapse into one candidate name, `Robert
|
||||
Drazkowski`). Disable this only if that assumption is wrong for a given
|
||||
pair of files - e.g. the trailing digit is genuinely disambiguating two
|
||||
different real people who happen to share a name, which would be
|
||||
unusual but isn't impossible.
|
||||
- `--manual "Speaker N=Real Name"` (repeatable) - force a specific label to
|
||||
a specific name directly, bypassing matching entirely. This is how you
|
||||
apply names the user supplies for labels the automatic pass couldn't
|
||||
resolve (see "After the preview" below) - it overrides any automatic
|
||||
result for that label, confident or not, and works even for a label
|
||||
that had zero automatic matches at all.
|
||||
- **Partial names are matched against the broken file's roster
|
||||
automatically.** If the user only gives a first name (or any partial
|
||||
string) and it matches exactly one real name already appearing in the
|
||||
broken file - e.g. `--manual "Speaker 3=Dawid"` when the roster
|
||||
contains `Dawid Cieślicki` and no one else called Dawid - the script
|
||||
expands it to the full roster name and prints a `NOTE (Speaker 3): ...`
|
||||
line saying so. Relay that note to the user so they know which full
|
||||
name actually got applied. If the partial name matches *more than one*
|
||||
roster name, it's genuinely ambiguous - the script keeps the literal
|
||||
string as given (does not guess) and prints a warning listing every
|
||||
candidate it could have meant; relay that to the user and ask them to
|
||||
supply the full name instead. If it matches *no one* in the roster, the
|
||||
literal name is used as-is with no note - that's the expected, normal
|
||||
case for a real participant who was never captured with a name
|
||||
anywhere in the broken file either (the roster-gap scenario), not an
|
||||
error.
|
||||
- `--examples <n>` (default 5) - how many timestamped example lines to
|
||||
print per label (spread across its timeline) for the user to spot-check
|
||||
against the actual recording. Raise it if a label needs more coverage
|
||||
before the user is comfortable confirming it.
|
||||
- `--recording-url <url>` - a link to the target meeting's recording (e.g.
|
||||
a Fireflies share URL). If given, it's printed once at the top and
|
||||
repeated under every label next to its example lines, so a reviewer has
|
||||
one click to the recording right where they need it. There's no verified
|
||||
way to encode an exact-timestamp deep link for Fireflies (their own UI
|
||||
has a "copy link to this moment" feature, but the query parameter it
|
||||
produces isn't publicly documented), so this links to the recording
|
||||
itself - the user still scrubs to each example's timestamp manually.
|
||||
|
||||
## Same speaker, two labels in the broken file
|
||||
|
||||
Fireflies sometimes emits two different name strings for what is actually
|
||||
one person, when its diarization isn't confident two speech segments are
|
||||
the same voice cluster - most visibly a trailing digit appended to an
|
||||
otherwise-identical name (`Robert Drazkowski` / `Robert Drazkowski1`). Left
|
||||
unhandled, this would split one real person's votes across two candidate
|
||||
names and could prevent either from reaching the resolution threshold, or
|
||||
worse, cause the script to treat them as two different (wrong) people.
|
||||
|
||||
The script merges these automatically before voting (`normalize_name`,
|
||||
default on) and **prints exactly what it merged** near the top of its
|
||||
output - always check that block. If it merged something that was actually
|
||||
two different people, or missed a variant that isn't a bare trailing digit
|
||||
(e.g. `Name (2)` or `Name_2` - the normalizer also handles these, but a
|
||||
truly unusual format might slip through), re-run with `--no-normalize` and
|
||||
handle that pair of names manually via `--manual` instead.
|
||||
|
||||
## After the preview (pass 1)
|
||||
|
||||
Report to the user, based on the script's own printed output, **before**
|
||||
ever running pass 2:
|
||||
- Which file it auto-detected as broken vs target - flag it if this looks
|
||||
wrong given the filenames/content.
|
||||
- Any merged duplicate-diarization labels (so they can sanity-check the
|
||||
merge was correct, not two different people collapsed into one).
|
||||
- The roster-gap warning, if any (how many target speakers can't
|
||||
structurally be resolved).
|
||||
- The offset it settled on.
|
||||
- The per-label resolution table (name/UNRESOLVED, vote count, confidence)
|
||||
- present this as proposed matches for the user to accept, not as a
|
||||
done deal.
|
||||
- **Any other candidates for a label, most to least probable.** Whenever a
|
||||
label had votes for more than one real name, the script prints a second
|
||||
line under it - `other candidates, most to least probable: ...` for a
|
||||
resolved label, `all candidates, most to least probable: ...` for an
|
||||
unresolved one - each with its own vote count and vote share. Always
|
||||
relay this ranked list, not just the winning name, especially when the
|
||||
top two candidates are close in vote share (e.g. 45% vs 40%): that's a
|
||||
near-tie, not a confident resolution, and the user may recognize the
|
||||
second-place name as the right one from the snippets.
|
||||
- **The example lines printed under every label** - the script picks up to
|
||||
`--examples` (default 5) lines per label, spread across that label's full
|
||||
timeline rather than clustered at the start, and prints each with its
|
||||
timestamp *in the target recording's own timeline* (not offset-adjusted -
|
||||
these are the timestamps to scrub to in the actual recording/video, since
|
||||
that's what the user has playback access to). This applies to every
|
||||
label, not just unresolved ones - relay them for the resolved labels too,
|
||||
and explicitly suggest the user jump to a couple of these timestamps in
|
||||
the recording and confirm by ear who's actually speaking, especially for
|
||||
anything under ~70% confidence. This is the concrete way to turn "the
|
||||
vote count says X" into "I checked and it's actually X" - don't skip
|
||||
offering it just because a label came back resolved. Raise `--examples`
|
||||
if the user wants more per label to check. If you have a link to the
|
||||
recording, pass it via `--recording-url` so it's printed alongside every
|
||||
label's examples - don't make the user go find the meeting themselves.
|
||||
- **When a `--manual` override is given, the automatic guess is still shown
|
||||
underneath it, not discarded** - the header line says whether the manual
|
||||
name *agrees with* the automatic guess, *overrides* it (naming what the
|
||||
automatic pass would have picked instead, and at what confidence), or
|
||||
fills a gap the automatic pass left unresolved. Relay this distinction -
|
||||
an override that contradicts a high-confidence automatic guess is worth
|
||||
flagging back to the user as a "you sure?" before applying, whereas one
|
||||
that just fills an unresolved gap or agrees with the automatic guess
|
||||
needs no extra scrutiny.
|
||||
- The minority-vote lines flagged for manual review, if any.
|
||||
- **If the roster-gap warning fired, look for suspicious patterns across
|
||||
multiple labels** before taking individual resolutions at face value -
|
||||
e.g. two different labels both resolving to the same real name, each at
|
||||
middling confidence with the same runner-up(s), is a sign that one of
|
||||
them is actually an unnamed real participant being misattributed, not
|
||||
genuinely two clusters of the same person. Flag this pattern explicitly
|
||||
to the user rather than reporting each label's line as independently
|
||||
fine.
|
||||
|
||||
Then explicitly recommend spot-checking the lowest-confidence resolutions
|
||||
against actual dialogue content - and, now that timestamps are available
|
||||
for every label, against the actual recording audio - before accepting
|
||||
them. Do not present the preview as a finished, verified transcript.
|
||||
|
||||
**Always relay the script's "Could not match" section and actually ask
|
||||
the user about it** - don't just print it and move on. For each
|
||||
unresolved label, show the line count and its example timestamps, then ask
|
||||
something like: "I couldn't match Speaker 2, 4, and 9 - here's what each
|
||||
said, with timestamps to check in the recording [examples]. Do you know
|
||||
who any of these are?"
|
||||
|
||||
**Wait for explicit confirmation before running pass 2 (`--apply`).** The
|
||||
user needs to either:
|
||||
- confirm the proposed matches look right, and/or
|
||||
- supply real names for any unresolved gaps, and/or
|
||||
- correct any match that looks wrong (even a "resolved" one they don't
|
||||
trust).
|
||||
|
||||
Fold whatever they give you into `--manual "Speaker N=Name"` flags (one
|
||||
per label) on the pass-2 run - this overrides the automatic result for
|
||||
that label, confident or not. Then re-run with `--apply` and show the
|
||||
updated report. Don't hand-edit the output file directly, since a re-run
|
||||
keeps the automatic resolutions and the merge/roster reporting consistent
|
||||
with the final file.
|
||||
|
||||
If the user wants a stricter or looser automatic pass instead, re-run
|
||||
pass 1 (still without `--apply`) with adjusted
|
||||
`--tolerance`/`--min-votes`/`--min-confidence` and present the new
|
||||
preview before applying anything.
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **Neither file matches the expected `**Speaker** *[MM:SS]*: text` format**
|
||||
(e.g. it's an `.mhtml` capture, not an extracted `.md`) - point the user
|
||||
at the `extract-transcript` skill first to get a proper Markdown export.
|
||||
- **A generic label appears in a burst with no broken-file activity nearby
|
||||
at all** (e.g. everyone else was silent while this person spoke for a
|
||||
while) - it will correctly come back `UNRESOLVED (no timestamp within
|
||||
tolerance found at all)` rather than a forced guess.
|
||||
- **The target file already has some real names mixed with `Speaker N`
|
||||
labels** (partial resolution done elsewhere) - only the `Speaker N`
|
||||
entries are touched; already-named lines are left exactly as they are.
|
||||
This doesn't confuse auto-detection either, since it compares the
|
||||
*share* of generic labels between the two files, not just their
|
||||
presence.
|
||||
- **Auto-detection can't tell the files apart** (exactly equal share of
|
||||
generic labels in both, e.g. both 0% or both 100%) - the script exits
|
||||
with an error rather than guessing, since swapping argument order
|
||||
wouldn't change the outcome either. Inspect both files by eye instead -
|
||||
it likely means one file isn't in the format expected, or this isn't
|
||||
actually a broken/target pair.
|
||||
- **Re-running after tuning flags or `--manual` overrides** - always safe
|
||||
in either pass. Without `--apply` nothing is ever written, and with
|
||||
`--apply` each run's output filename defaults to the same path
|
||||
(overwritten on re-run, not accumulated).
|
||||
- **`--manual` references a label that doesn't exist in the target** (typo,
|
||||
or a label that already has a real name) - the script warns and ignores
|
||||
it rather than silently doing nothing; check the warning if a manual
|
||||
override doesn't seem to have taken effect.
|
||||
- **A `--manual` partial name matches more than one roster name** (e.g. two
|
||||
different real people in the broken file share a first name) - the
|
||||
script refuses to guess, uses the literal string as given, and prints a
|
||||
warning listing every candidate it could have meant. Relay this to the
|
||||
user and get the full name before applying, rather than letting the
|
||||
ambiguous literal string silently become the final label.
|
||||
480
.agents/skills/transcript-speaker-fill/scripts/fill_speakers.py
Normal file
480
.agents/skills/transcript-speaker-fill/scripts/fill_speakers.py
Normal file
|
|
@ -0,0 +1,480 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Fill in real speaker names in a Fireflies-style transcript by fuzzy-matching
|
||||
timestamps against a second transcript of the same meeting that has real
|
||||
speaker names but broken/unusable text (e.g. wrong language was detected,
|
||||
so the words are garbage but the diarization + speaker labels are fine).
|
||||
|
||||
Both inputs are expected in Fireflies' Markdown export shape:
|
||||
**Speaker Name** *[MM:SS]*: text
|
||||
|
||||
The two file arguments can be given in either order - whichever one has a
|
||||
higher share of generic "Speaker N" labels is auto-detected as the target
|
||||
to fill in; the other is treated as the broken source of real names.
|
||||
|
||||
By default this is a dry run: it only prints the matching report. Nothing
|
||||
is written until it is re-run with --apply, so the report can be reviewed
|
||||
(and --manual overrides added for any gaps) before anything touches disk.
|
||||
|
||||
Deterministic, no model reasoning involved - see the accompanying SKILL.md
|
||||
for when/how to invoke this.
|
||||
"""
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from collections import defaultdict, Counter
|
||||
|
||||
LINE_RE = re.compile(
|
||||
r'^\*\*(?P<speaker>[^*]+)\*\*\s*\*\[(?P<ts>\d{1,2}:\d{2}(?::\d{2})?)\]\*:\s*(?P<text>.*)$'
|
||||
)
|
||||
GENERIC_RE = re.compile(r'^Speaker\s+\d+$')
|
||||
|
||||
|
||||
def parse_ts(ts):
|
||||
parts = [int(p) for p in ts.split(':')]
|
||||
if len(parts) == 2:
|
||||
m, s = parts
|
||||
return m * 60 + s
|
||||
h, m, s = parts
|
||||
return h * 3600 + m * 60 + s
|
||||
|
||||
|
||||
def normalize_name(name):
|
||||
"""Fireflies sometimes labels the same real person with two different
|
||||
strings when it isn't sure two segments are the same voice cluster -
|
||||
most commonly a trailing digit appended to an otherwise-identical name
|
||||
(e.g. "Robert Drazkowski" and "Robert Drazkowski1" are the same person).
|
||||
Strip that suffix so both collapse into one candidate name for voting.
|
||||
Also strips a trailing " 2", "(2)", "_2" etc. in case Fireflies uses one
|
||||
of those variants instead of a bare digit."""
|
||||
cleaned = re.sub(r'[\s_]*\(?\d+\)?$', '', name).strip()
|
||||
return cleaned if cleaned else name.strip()
|
||||
|
||||
|
||||
def parse_manual_overrides(pairs):
|
||||
"""Parse repeated --manual "Speaker N=Real Name" arguments into a dict."""
|
||||
overrides = {}
|
||||
for pair in pairs or []:
|
||||
if '=' not in pair:
|
||||
sys.exit(f"--manual expects 'Speaker N=Real Name', got: {pair!r}")
|
||||
label, name = pair.split('=', 1)
|
||||
overrides[label.strip()] = name.strip()
|
||||
return overrides
|
||||
|
||||
|
||||
def resolve_partial_name(name, roster):
|
||||
"""If `name` is a partial reference (e.g. just a first name) to someone
|
||||
who already appears in `roster` (the broken file's real-name roster),
|
||||
suggest/expand to the matching full name instead of taking the partial
|
||||
string literally. Returns (final_name, note_or_None).
|
||||
|
||||
Matching is deliberately conservative: an exact match short-circuits
|
||||
immediately; otherwise a candidate qualifies only if `name` shares a
|
||||
whole word with it (case-insensitive) or is a substring of it. If more
|
||||
than one roster name qualifies, this is ambiguous - the literal name is
|
||||
kept as given rather than guessing, with a warning listing the
|
||||
candidates so the user can specify which one they meant. If none
|
||||
qualify, the name is genuinely new (e.g. a real participant who was
|
||||
never captured with a name anywhere in the broken file) and is used
|
||||
as-is - that's expected, not an error."""
|
||||
if name in roster:
|
||||
return name, None
|
||||
|
||||
name_lower = name.strip().lower()
|
||||
name_tokens = set(name_lower.split())
|
||||
candidates = []
|
||||
for full in roster:
|
||||
full_lower = full.lower()
|
||||
if name_lower == full_lower:
|
||||
return full, f"'{name}' matches roster name '{full}' (case-insensitive) - using '{full}'."
|
||||
full_tokens = set(full_lower.split())
|
||||
if name_tokens & full_tokens or name_lower in full_lower:
|
||||
candidates.append(full)
|
||||
|
||||
if len(candidates) == 1:
|
||||
return candidates[0], f"'{name}' looks like a partial name - matched to the only candidate in the roster, '{candidates[0]}'. Using the full name."
|
||||
if len(candidates) > 1:
|
||||
return name, (
|
||||
f"'{name}' is ambiguous - it could refer to any of: {', '.join(candidates)}. "
|
||||
f"Used literally as given since I can't tell which one you meant - re-run with the full "
|
||||
f"name to disambiguate if this isn't who you intended."
|
||||
)
|
||||
return name, None
|
||||
|
||||
|
||||
def parse_file(path):
|
||||
entries = []
|
||||
for i, line in enumerate(Path(path).read_text(encoding='utf-8').splitlines()):
|
||||
m = LINE_RE.match(line.strip())
|
||||
if m:
|
||||
entries.append({
|
||||
'line_no': i,
|
||||
'speaker': m.group('speaker').strip(),
|
||||
'seconds': parse_ts(m.group('ts')),
|
||||
'text': m.group('text'),
|
||||
})
|
||||
return entries
|
||||
|
||||
|
||||
def classify(entries_a, path_a, entries_b, path_b):
|
||||
"""Decide which of the two parsed transcripts is the 'target' (has
|
||||
generic Speaker N labels needing real names filled in) and which is
|
||||
the 'broken' source (has real names already, used only for its
|
||||
timestamps). Whichever file has a higher share of generic-labeled
|
||||
lines is the target - the broken source should have few or none,
|
||||
since its diarization already resolved real names even though its
|
||||
text is garbled. This replaces any assumption about file size or
|
||||
argument order."""
|
||||
def generic_fraction(entries):
|
||||
if not entries:
|
||||
return 0.0
|
||||
generic = sum(1 for e in entries if GENERIC_RE.match(e['speaker']))
|
||||
return generic / len(entries)
|
||||
|
||||
frac_a = generic_fraction(entries_a)
|
||||
frac_b = generic_fraction(entries_b)
|
||||
|
||||
if frac_a == frac_b:
|
||||
sys.exit(
|
||||
f"Could not automatically tell which file needs speaker names filled in: "
|
||||
f"both '{path_a}' and '{path_b}' have the same share of generic 'Speaker N' "
|
||||
f"labels ({frac_a:.0%}). Check the files by eye - the target should read as "
|
||||
f"real dialogue with generic labels, the broken source should have real names "
|
||||
f"but garbled/wrong-language text."
|
||||
)
|
||||
|
||||
if frac_a > frac_b:
|
||||
return (entries_a, path_a), (entries_b, path_b)
|
||||
return (entries_b, path_b), (entries_a, path_a)
|
||||
|
||||
|
||||
def proximity_weight(delta, tolerance):
|
||||
"""1.0 for an exact match, decaying linearly to just above 0 at the
|
||||
tolerance boundary. A match a few seconds off is real signal; a match
|
||||
12 seconds off inside a 15s tolerance is mostly noise - weight
|
||||
accordingly rather than counting both as one equal 'vote'."""
|
||||
return max(0.0, 1.0 - delta / (tolerance + 1))
|
||||
|
||||
|
||||
def find_best_offset(anchor_entries, target_entries, tolerance, offset_range):
|
||||
"""Search for a constant clock offset (seconds) between the two
|
||||
recordings that maximizes total proximity-weighted overlap between
|
||||
target and anchor timestamps. Handles the two Fireflies bots not
|
||||
starting at exactly the same instant. Weighted (not a raw count of
|
||||
"any match within tolerance") so a wide tolerance can't let a wrong
|
||||
offset win just by picking up many loose, low-quality matches."""
|
||||
anchor_times = [e['seconds'] for e in anchor_entries]
|
||||
best_offset, best_score = 0, -1.0
|
||||
for offset in range(-offset_range, offset_range + 1):
|
||||
score = 0.0
|
||||
for t in target_entries:
|
||||
tt = t['seconds'] + offset
|
||||
best_delta = min((abs(tt - at) for at in anchor_times), default=tolerance + 1)
|
||||
score += proximity_weight(best_delta, tolerance)
|
||||
if score > best_score or (score == best_score and abs(offset) < abs(best_offset)):
|
||||
best_score, best_offset = score, offset
|
||||
return best_offset, best_score
|
||||
|
||||
|
||||
def nearest_match(seconds, anchor_entries, tolerance):
|
||||
best, best_delta = None, tolerance + 1
|
||||
for a in anchor_entries:
|
||||
delta = abs(a['seconds'] - seconds)
|
||||
if delta <= tolerance and delta < best_delta:
|
||||
best_delta, best = delta, a
|
||||
return best, best_delta
|
||||
|
||||
|
||||
def sample_examples(entries, n):
|
||||
"""Pick up to n example entries spread across the full span of entries
|
||||
(not just the first n) so a spot-check sees variety across the
|
||||
meeting's timeline rather than one early cluster."""
|
||||
if not entries:
|
||||
return []
|
||||
if len(entries) <= n:
|
||||
return entries
|
||||
if n <= 1:
|
||||
return [entries[0]]
|
||||
idxs = sorted({round(i * (len(entries) - 1) / (n - 1)) for i in range(n)})
|
||||
return [entries[i] for i in idxs]
|
||||
|
||||
|
||||
def format_examples(entries, indent=' '):
|
||||
lines = []
|
||||
for e in entries:
|
||||
mm, ss = divmod(e['seconds'], 60)
|
||||
snippet = e['text'].strip()
|
||||
if len(snippet) > 90:
|
||||
snippet = snippet[:90].rstrip() + "..."
|
||||
lines.append(f"{indent}[{mm:02d}:{ss:02d}] {snippet}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('file_a', help='One of the two transcripts (either order - the broken/target roles are auto-detected)')
|
||||
ap.add_argument('file_b', help='The other transcript')
|
||||
ap.add_argument('--apply', action='store_true',
|
||||
help='Write the output file. Without this flag, only the matching report is '
|
||||
'printed (dry run) so you can review and confirm before anything is written.')
|
||||
ap.add_argument('--tolerance', type=int, default=15,
|
||||
help='Max seconds between matched timestamps (default: 15)')
|
||||
ap.add_argument('--offset-search', type=int, default=90,
|
||||
help='Search +/- this many seconds for a global clock offset between the two recordings (default: 90)')
|
||||
ap.add_argument('--min-votes', type=int, default=2,
|
||||
help='Minimum matched votes required to resolve a label (default: 2)')
|
||||
ap.add_argument('--min-confidence', type=float, default=0.5,
|
||||
help='Minimum vote share (0-1) required to resolve a label (default: 0.5)')
|
||||
ap.add_argument('--output', help='Output path (default: <target-stem>-speakers-filled.md next to target)')
|
||||
ap.add_argument('--no-normalize', action='store_true',
|
||||
help='Do not merge duplicate-diarization name variants (e.g. "Name" / "Name1") in the broken file before voting')
|
||||
ap.add_argument('--manual', action='append', metavar='"Speaker N=Real Name"',
|
||||
help='Force a specific label to a specific name, bypassing matching entirely. '
|
||||
'Repeatable. Overrides any automatic result (confident or not) for that label.')
|
||||
ap.add_argument('--examples', type=int, default=5,
|
||||
help='Example lines (with timestamps) to print per label, spread across its full '
|
||||
'timeline, so matches can be spot-checked against the actual recording (default: 5)')
|
||||
ap.add_argument('--recording-url',
|
||||
help="Link to the target meeting's recording (e.g. a Fireflies share URL). If given, "
|
||||
"it's printed once at the top and repeated under every label so a reviewer can "
|
||||
"jump straight to it and scrub to each example's timestamp. There is no verified "
|
||||
"way to encode an exact-timestamp deep link for Fireflies (the parameter their "
|
||||
"own UI uses for 'copy link to this moment' isn't publicly documented), so this "
|
||||
"links to the recording itself, not a specific moment in it.")
|
||||
args = ap.parse_args()
|
||||
|
||||
manual_overrides = parse_manual_overrides(args.manual)
|
||||
|
||||
entries_a = parse_file(args.file_a)
|
||||
entries_b = parse_file(args.file_b)
|
||||
|
||||
if not entries_a:
|
||||
sys.exit(f"No '**Speaker** *[MM:SS]*: text' lines found in {args.file_a} - check the format.")
|
||||
if not entries_b:
|
||||
sys.exit(f"No '**Speaker** *[MM:SS]*: text' lines found in {args.file_b} - check the format.")
|
||||
|
||||
(target_entries, target_path), (broken_entries, broken_path) = classify(
|
||||
entries_a, args.file_a, entries_b, args.file_b
|
||||
)
|
||||
print(f"Auto-detected roles: '{broken_path}' has real names (broken source), "
|
||||
f"'{target_path}' has generic labels to fill (target).")
|
||||
print()
|
||||
|
||||
merged_variants = defaultdict(set)
|
||||
if not args.no_normalize:
|
||||
for b in broken_entries:
|
||||
raw = b['speaker']
|
||||
normalized = normalize_name(raw)
|
||||
if normalized != raw:
|
||||
merged_variants[normalized].add(raw)
|
||||
b['speaker'] = normalized
|
||||
|
||||
if merged_variants:
|
||||
print("Merged duplicate diarization labels in the broken file (treated as one person):")
|
||||
for normalized, raws in sorted(merged_variants.items()):
|
||||
variants = sorted(raws | {normalized})
|
||||
print(f" {' / '.join(variants)} -> {normalized}")
|
||||
print()
|
||||
|
||||
broken_roster = sorted(set(b['speaker'] for b in broken_entries))
|
||||
target_generic_labels = sorted(set(t['speaker'] for t in target_entries if GENERIC_RE.match(t['speaker'])))
|
||||
if len(target_generic_labels) > len(broken_roster):
|
||||
gap = len(target_generic_labels) - len(broken_roster)
|
||||
print(f"NOTE: the target has {len(target_generic_labels)} distinct unnamed speakers but the broken "
|
||||
f"file only names {len(broken_roster)} real people ({', '.join(broken_roster)}). At least "
|
||||
f"{gap} target speaker(s) are structurally impossible to resolve correctly - they (or their "
|
||||
f"voice segments) simply aren't captured with a real name anywhere in the broken file, so "
|
||||
f"the closest-timestamp match for them, if any, will be a coincidence, not a correspondence. "
|
||||
f"Treat any resolution below with real skepticism, especially ones with few votes.")
|
||||
print()
|
||||
|
||||
offset, offset_score = find_best_offset(broken_entries, target_entries, args.tolerance, args.offset_search)
|
||||
|
||||
weighted_votes = defaultdict(Counter) # label -> name -> summed proximity weight
|
||||
raw_votes = defaultdict(Counter) # label -> name -> raw match count (for the min-votes gate)
|
||||
minority_lines = []
|
||||
for t in target_entries:
|
||||
if not GENERIC_RE.match(t['speaker']):
|
||||
continue
|
||||
match, delta = nearest_match(t['seconds'] + offset, broken_entries, args.tolerance)
|
||||
if match:
|
||||
weighted_votes[t['speaker']][match['speaker']] += proximity_weight(delta, args.tolerance)
|
||||
raw_votes[t['speaker']][match['speaker']] += 1
|
||||
t['_matched_name'] = match['speaker']
|
||||
t['_matched_delta'] = delta
|
||||
|
||||
resolution = {}
|
||||
for label, counter in weighted_votes.items():
|
||||
total_weight = sum(counter.values())
|
||||
ranked = counter.most_common() # all candidates, highest weighted vote share first
|
||||
name, top_weight = ranked[0]
|
||||
top_count = raw_votes[label][name]
|
||||
confidence = top_weight / total_weight if total_weight else 0.0
|
||||
resolved = top_count >= args.min_votes and confidence >= args.min_confidence
|
||||
candidates = [
|
||||
{
|
||||
'name': cand_name,
|
||||
'raw_count': raw_votes[label][cand_name],
|
||||
'share': weight / total_weight if total_weight else 0.0,
|
||||
}
|
||||
for cand_name, weight in ranked
|
||||
]
|
||||
resolution[label] = {
|
||||
'name': name if resolved else None,
|
||||
'top_count': top_count,
|
||||
'total_votes': sum(raw_votes[label].values()),
|
||||
'confidence': confidence,
|
||||
'candidates': candidates,
|
||||
}
|
||||
if resolved:
|
||||
for t in target_entries:
|
||||
if t['speaker'] == label and t.get('_matched_name') and t['_matched_name'] != name:
|
||||
minority_lines.append((t['line_no'], t['seconds'], label, t['_matched_name'], name))
|
||||
|
||||
unknown_manual_labels = [label for label in manual_overrides if label not in target_generic_labels]
|
||||
if unknown_manual_labels:
|
||||
print(f"WARNING: --manual referenced label(s) not found as a generic speaker in the target file: "
|
||||
f"{', '.join(unknown_manual_labels)} - ignoring them. Known generic labels: "
|
||||
f"{', '.join(target_generic_labels)}")
|
||||
print()
|
||||
|
||||
# Resolve any partial names in --manual against the broken file's real-name
|
||||
# roster (e.g. "Dawid" -> "Dawid Cieślicki" if that's the only roster match),
|
||||
# rather than taking the literal string when a better match is available.
|
||||
manual_final = {}
|
||||
for label, raw_name in manual_overrides.items():
|
||||
if label not in target_generic_labels:
|
||||
continue
|
||||
resolved_name, note = resolve_partial_name(raw_name, broken_roster)
|
||||
manual_final[label] = resolved_name
|
||||
if note:
|
||||
print(f"NOTE ({label}): {note}")
|
||||
if manual_final:
|
||||
print()
|
||||
|
||||
# Automatic `resolution` is left untouched here (it stays the source of
|
||||
# truth for the automatic guess/candidates, printed for every label
|
||||
# below regardless of whether a manual override wins); `final_name` is
|
||||
# what actually gets written to disk.
|
||||
final_name = {}
|
||||
for label in target_generic_labels:
|
||||
info = resolution.get(label)
|
||||
final_name[label] = info['name'] if info else None
|
||||
final_name.update(manual_final)
|
||||
|
||||
lines = Path(target_path).read_text(encoding='utf-8').splitlines()
|
||||
resolved_line_count = 0
|
||||
generic_totals = Counter(t['speaker'] for t in target_entries if GENERIC_RE.match(t['speaker']))
|
||||
for t in target_entries:
|
||||
if not GENERIC_RE.match(t['speaker']):
|
||||
continue
|
||||
name = final_name.get(t['speaker'])
|
||||
if name:
|
||||
old = f"**{t['speaker']}**"
|
||||
new = f"**{name}**"
|
||||
lines[t['line_no']] = lines[t['line_no']].replace(old, new, 1)
|
||||
resolved_line_count += 1
|
||||
|
||||
out_path = Path(args.output) if args.output else Path(target_path).with_name(
|
||||
Path(target_path).stem + "-speakers-filled.md"
|
||||
)
|
||||
if args.apply:
|
||||
out_path.write_text("\n".join(lines) + "\n", encoding='utf-8')
|
||||
|
||||
total_generic_lines = sum(generic_totals.values())
|
||||
resolved_labels = sum(1 for name in final_name.values() if name)
|
||||
|
||||
print(f"Global offset applied: {offset:+d}s (best fit: {offset_score}/{len(target_entries)} target lines matched at that offset)")
|
||||
if args.apply:
|
||||
print(f"Output written to: {out_path}")
|
||||
else:
|
||||
print(f"DRY RUN - no file written. Would write to: {out_path}")
|
||||
if args.recording_url:
|
||||
print(f"Recording: {args.recording_url}")
|
||||
print("(no verified way to deep-link an exact timestamp - open this and scrub to each example below)")
|
||||
print()
|
||||
print("Speaker label resolution:")
|
||||
unresolved_labels = []
|
||||
for label in sorted(generic_totals, key=lambda l: -generic_totals[l]):
|
||||
info = resolution.get(label)
|
||||
manual_name = manual_final.get(label)
|
||||
n_lines = generic_totals[label]
|
||||
label_entries = [t for t in target_entries if t['speaker'] == label]
|
||||
examples = sample_examples(label_entries, args.examples)
|
||||
|
||||
# Header line: the FINAL decision for this label (manual wins if given).
|
||||
if manual_name:
|
||||
agreement = ""
|
||||
if info and info.get('name') == manual_name:
|
||||
agreement = " - agrees with automatic guess"
|
||||
elif info and info.get('name'):
|
||||
agreement = f" - OVERRIDES automatic guess of '{info['name']}' ({info['confidence']:.0%} confidence)"
|
||||
elif info:
|
||||
agreement = " - automatic pass left this unresolved"
|
||||
print(f" {label:12s} -> {manual_name:25s} (manually provided{agreement}) - {n_lines} lines")
|
||||
elif info is None:
|
||||
print(f" {label:12s} -> UNRESOLVED (no timestamp within tolerance found at all) - {n_lines} lines")
|
||||
unresolved_labels.append(label)
|
||||
elif info['name']:
|
||||
print(f" {label:12s} -> {info['name']:25s} ({info['top_count']}/{info['total_votes']} votes, "
|
||||
f"{info['confidence']:.0%} confidence) - {n_lines} lines")
|
||||
else:
|
||||
print(f" {label:12s} -> UNRESOLVED (top guess {info['top_count']}/{info['total_votes']} votes, "
|
||||
f"{info['confidence']:.0%} confidence, below threshold) - {n_lines} lines")
|
||||
unresolved_labels.append(label)
|
||||
|
||||
# All candidates with their confidence, for every label regardless of
|
||||
# whether the final answer came from automatic matching or --manual -
|
||||
# so a manual override's plausibility can still be judged against
|
||||
# what the timestamps alone suggested.
|
||||
if info and info.get('candidates'):
|
||||
winner = info.get('name')
|
||||
others = [c for c in info['candidates'] if c['name'] != winner] if winner else info['candidates']
|
||||
label_str = "other candidates" if winner else "all candidates"
|
||||
if others:
|
||||
ranked_str = ", ".join(
|
||||
f"{c['name']} ({c['raw_count']}/{info['total_votes']} votes, {c['share']:.0%})"
|
||||
for c in others
|
||||
)
|
||||
print(f" {label_str}, most to least probable: {ranked_str}")
|
||||
|
||||
if examples:
|
||||
print(f" example lines (jump to these timestamps in the recording to verify):")
|
||||
print(format_examples(examples))
|
||||
if args.recording_url:
|
||||
print(f" recording: {args.recording_url}")
|
||||
print()
|
||||
print(f"Resolved {resolved_labels}/{len(generic_totals)} distinct generic labels, "
|
||||
f"covering {resolved_line_count}/{total_generic_lines} generic-labeled lines.")
|
||||
|
||||
if minority_lines:
|
||||
print()
|
||||
print(f"{len(minority_lines)} individual line(s) disagreed with their label's majority vote "
|
||||
f"(kept the majority name, flagging for manual review):")
|
||||
for line_no, seconds, label, minority_name, majority_name in minority_lines[:20]:
|
||||
mm, ss = divmod(seconds, 60)
|
||||
print(f" line {line_no + 1} [{mm:02d}:{ss:02d}] {label}: nearest match was "
|
||||
f"'{minority_name}', used majority '{majority_name}' instead")
|
||||
if len(minority_lines) > 20:
|
||||
print(f" ... and {len(minority_lines) - 20} more")
|
||||
|
||||
if unresolved_labels:
|
||||
print()
|
||||
print("=" * 70)
|
||||
print(f"Could not match {len(unresolved_labels)} speaker(s): {', '.join(unresolved_labels)} "
|
||||
f"(see their example lines/timestamps above).")
|
||||
print("=" * 70)
|
||||
print("If you know who any of these are, provide their real names and re-run with, e.g.:")
|
||||
example = unresolved_labels[0]
|
||||
print(f' --manual "{example}=Real Name"' + (' --manual "..."' if len(unresolved_labels) > 1 else ''))
|
||||
|
||||
if not args.apply:
|
||||
print()
|
||||
print("-" * 70)
|
||||
print("DRY RUN - nothing was written. Review the resolution table above (and any "
|
||||
"unresolved gaps), add --manual \"Speaker N=Real Name\" for anything to correct "
|
||||
"or fill in, then re-run with --apply to write the output file.")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
6
.claude/settings.json
Normal file
6
.claude/settings.json
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
]
|
||||
}
|
||||
}
|
||||
6
.gitignore
vendored
6
.gitignore
vendored
|
|
@ -1,2 +1,8 @@
|
|||
libs/
|
||||
tmp/
|
||||
outputs/starlight
|
||||
outputs/okf
|
||||
|
||||
.claude/skills
|
||||
|
||||
.env
|
||||
|
|
@ -23,7 +23,9 @@ The root directory contains exactly seven top-level entries. You must maintain t
|
|||
│ # On-demand workflows beyond Ingest/Lint may be defined as Claude Code Skills under
|
||||
│ # `.claude/skills/` — check there before assuming a capability doesn't exist.
|
||||
├── raw/ # WRITTEN BY USER ONLY. Raw files, scratchpad notes, URLs, links.txt.
|
||||
│ └── inbox/ # Drop zone: unprocessed material the agent cleans on ingest.
|
||||
│ ├── inbox/ # Drop zone: unprocessed material the agent cleans on ingest.
|
||||
│ └── archive/ # AGENT MAINTAINED. Ingested raw material, filed by ingestion date.
|
||||
│ └── <YYYY-MM-DD>/ # One folder per ingestion date; holds every raw/inbox file processed that day.
|
||||
├── tmp/ # MANAGED BY AGENT. Temporary files, caches, intermediate processing artifacts (gitignored).
|
||||
├── wiki/ # MANAGED BY AGENT. The local, mutable, structured markdown wiki. Overlays linked/ and libs/.
|
||||
│ ├── index.md # Entry point / routing table with "Use when" triggers. Carries kb_schema_version.
|
||||
|
|
@ -115,7 +117,7 @@ retention: high|medium|low # How aggressively to deprioritize when old
|
|||
|
||||
When the user says "Ingest", "Sync the wiki", or "Update the Wiki" (for syncing this repo's own git history with its remote, see the sync-changes skill under `.claude/skills/` instead):
|
||||
|
||||
1. **Process Inbox:** Scan `raw/inbox/` for new material. Move processed items to `raw/` for archiving. If `raw/inbox/` is empty, scan `raw/` directly.
|
||||
1. **Process Inbox:** Scan `raw/inbox/` for new material. After ingesting, move each processed item to `raw/archive/<YYYY-MM-DD>/`, where the date is today's ingestion date (create the dated folder if it doesn't exist yet). If `raw/inbox/` is empty, scan `raw/` directly (excluding `raw/archive/`, which holds already-processed material).
|
||||
|
||||
2. **Consult Cascade:** Before writing anything, check if the entity already exists (local `wiki/` first, then each `linked/<name>/`, then each `libs/<name>/`). The local `wiki/` always wins. Upstream content is informative but can be overridden locally.
|
||||
|
||||
|
|
|
|||
225
CLAUDE.md
225
CLAUDE.md
|
|
@ -1,225 +0,0 @@
|
|||
# SYSTEM PROMPT: CASCADE KNOWLEDGE BASE ARCHITECT
|
||||
|
||||
## ROLE & PHILOSOPHY
|
||||
You are an autonomous Knowledge Architect Agent for a **Cascade Knowledge Base**. The system is designed as a layered stack: read-only upstream knowledge bases (symlinked in `linked/` and git-managed copies in `libs/`) form the foundation, and the local mutable knowledge base overlays on top. This means knowledge flows downward through the cascade — upstream truths are preserved, while you only ever modify the local layer.
|
||||
|
||||
If an entity exists in both the local wiki and any upstream KB, the local version takes precedence and overrides the upstream one.
|
||||
|
||||
You view directories as storage disks, context windows as RAM, and your processing loops as CPU cycles. Your sole objective is to build, maintain, and dynamically structure a comprehensive knowledge base, respecting the cascade priority rules at all times.
|
||||
|
||||
You possess full autonomy over local directory structure, file naming conventions, and cross-referencing. You must strictly adhere to the operational boundaries and file management rules detailed below.
|
||||
|
||||
---
|
||||
|
||||
## 1. DIRECTORY STRUCTURE
|
||||
The root directory contains exactly seven top-level entries. You must maintain this structure flawlessly:
|
||||
|
||||
```
|
||||
├── libs/ # GIT-MANAGED COPIES ONLY. Read-only external KBs copied into the repo (gitignored — populated by the user via git).
|
||||
│ └── <name>/ # Individual external knowledge base (immutable — never write here).
|
||||
├── linked/ # SYMLINKS ONLY. Each entry is a symbolic link to another KB root (read-only upstream source of truth).
|
||||
│ └── <name>/ # Individual upstream knowledge base (immutable — never write here).
|
||||
├── outputs/ # MANAGED BY AGENT. Generated artifacts, exports, compiled files produced from the wiki.
|
||||
│ # On-demand workflows beyond Ingest/Lint may be defined as Claude Code Skills under
|
||||
│ # `.claude/skills/` — check there before assuming a capability doesn't exist.
|
||||
├── raw/ # WRITTEN BY USER ONLY. Raw files, scratchpad notes, URLs, links.txt.
|
||||
│ └── inbox/ # Drop zone: unprocessed material the agent cleans on ingest.
|
||||
├── tmp/ # MANAGED BY AGENT. Temporary files, caches, intermediate processing artifacts (gitignored).
|
||||
├── wiki/ # MANAGED BY AGENT. The local, mutable, structured markdown wiki. Overlays linked/ and libs/.
|
||||
│ ├── index.md # Entry point / routing table with "Use when" triggers. Carries kb_schema_version.
|
||||
│ ├── overview.md # High-level map of the knowledge base.
|
||||
│ ├── log.md # AGENT LOG. Root rollup tracking wiki-level modifications (see Recursive Index & Log Convention).
|
||||
│ ├── error-book.md # AGENT MAINTAINED. Records compilation errors and derived constraints.
|
||||
│ ├── entities/ # AGENT POPULATED. Typed entity pages (people, projects, concepts, libraries). Has its own index.md.
|
||||
│ └── graph/ # AGENT MAINTAINED. Edge lists and relationship data for the knowledge graph. Has its own index.md.
|
||||
└── workload/ # MANAGED BY AGENT. Summaries of discussions and decisions.
|
||||
└── YYYY-MM-DD_summary.md
|
||||
```
|
||||
|
||||
### Cascade Lookup Priority
|
||||
When searching for any entity, concept, or file, use the following cascade (first match wins):
|
||||
|
||||
1. **Local wiki/** — highest priority; agent-written content overlays everything below.
|
||||
2. **linked/\<name\>/** — read-only upstream KBs mounted as symlinks, searched in alphabetical order.
|
||||
3. **libs/\<name\>/** — read-only external KBs copied via git, searched in alphabetical order.
|
||||
4. If no match is found anywhere, treat the entity as unknown.
|
||||
|
||||
You must **never** create, modify, move, or delete any file or directory inside `linked/` or `libs/`.
|
||||
|
||||
### Index-First Navigation
|
||||
When searching for information, always start by looking for `index.md` files.
|
||||
Read the index to discover what pages and subdirectories are available before
|
||||
drilling into individual files. Scan `index.md` across all layers:
|
||||
|
||||
1. **wiki/** — scan `wiki/index.md`, then recursively check any subdirectory `wiki/<topic>/index.md`.
|
||||
2. **linked/\<name\>/** — for each linked upstream KB, scan its root `index.md` and subdirectory indexes.
|
||||
3. **libs/\<name\>/** — same pattern: root index first, then subdirectory indexes as needed.
|
||||
|
||||
This avoids blind filesystem scans and uses the index as a curated table of contents — exactly as Karpathy's original pattern intended.
|
||||
|
||||
### Recursive Index & Log Convention
|
||||
Index-First Navigation only works if subdirectory indexes actually exist. Maintain them as follows:
|
||||
|
||||
- Every `wiki/` subdirectory that groups multiple pages (`entities/`, `graph/`, and any future topic folder) must contain its own `index.md`. It carries no frontmatter and is a flat bullet list of links, each with a one-line description mirroring the linked page's `tldr` — plus a link to any nested subdirectory.
|
||||
- A subdirectory may also keep its own `log.md` once it has enough independent change history to warrant one (a judgment call — typically once it holds several pages or changes on its own cadence, separate from the rest of the wiki). Entries follow the same reverse-chronological format as Rule B.
|
||||
- The root `wiki/log.md` stays the top-level rollup: it records changes made directly under `wiki/` (`index.md`, `overview.md`, `error-book.md`, directory-creation events) plus one pointer line whenever a subdirectory log absorbs a change, e.g. `- See wiki/entities/log.md for entity-page changes on this date.` Each change gets exactly one home log — never record the same change in both.
|
||||
|
||||
### Lazy-Loading with "Use When" Triggers
|
||||
The `wiki/index.md` is a routing table. Each entry has a **Use when** column
|
||||
that lists trigger keywords. Before loading any page:
|
||||
|
||||
1. Read `wiki/index.md` (stays in context — it is small).
|
||||
2. Match the current task's keywords against the **Use when** entries.
|
||||
3. Only load the matching page(s). Do not load every page.
|
||||
4. If a page has a `tldr:` frontmatter field, read that first. If it answers the query, skip the body.
|
||||
|
||||
This keeps context lean: ~3–4 pages loaded instead of all pages.
|
||||
|
||||
---
|
||||
|
||||
## 2. PAGE FRONTMATTER SCHEMA
|
||||
|
||||
Every wiki page must use YAML frontmatter. `type` is required; the rest are optional:
|
||||
|
||||
```yaml
|
||||
---
|
||||
type: concept # REQUIRED. Open string for the entity/content kind (e.g. person, project, concept, library, decision, playbook). Unregistered — new values are always valid; readers must tolerate unrecognized types.
|
||||
resource: https://... # Optional. Canonical URI to the authoritative external source this page describes (a linked/<name>/... or libs/<name>/... path, ticket, repo, doc, dataset). Keeps "what the wiki says about it" separate from "where the real thing lives."
|
||||
tldr: One-sentence summary optimised for LLM reading
|
||||
confidence: 0.0–1.0 # How many/corroborated sources support this
|
||||
quality: 0.0–1.0 # Self-evaluation: well-structured, consistent, cited
|
||||
supersedes: path/to/older/page.md
|
||||
superseded_by: path/to/newer/page.md
|
||||
last_updated: YYYY-MM-DD
|
||||
freshness_window_days: 90 # Days before considered potentially stale
|
||||
retention: high|medium|low # How aggressively to deprioritize when old
|
||||
---
|
||||
```
|
||||
|
||||
- **`type`** — required on every page. Set once on write and rarely changed; it's the first thing lint checks for conformance, and it's how pages in `entities/` get grouped without depending on directory naming alone.
|
||||
- **`resource`** — set when the page describes something with a stable external address. Omit for pages that are pure synthesis (e.g. an overview or a decision writeup with no single external source).
|
||||
- **`tldr`** — generated on write. If the TLDR alone answers a query, the body is never loaded.
|
||||
- **`confidence`** — set on write based on source corroboration. Decays with time unless reinforced by new sources.
|
||||
- **`quality`** — self-score on write. Below 0.7 → flag for review.
|
||||
- **`supersedes` / `superseded_by`** — when new info contradicts or updates an old page, link them. Old pages are preserved but marked stale.
|
||||
- **`last_updated`** — set automatically on every write or edit.
|
||||
- **`freshness_window_days`** — pages older than this window are flagged stale during lint.
|
||||
- **`retention`** — `low` pages may be archived or deprioritized after the freshness window expires.
|
||||
|
||||
### Schema Versioning
|
||||
`wiki/index.md` (only) carries an additional frontmatter field, `kb_schema_version` (e.g. `"1.1"`), declaring which revision of this schema the wiki was authored against. Bump the minor version when adding an optional field (backward-compatible); bump the major version when changing or removing a required field or reserved filename convention (breaking). Individual pages do not carry this field — it is a bundle-level declaration, not a per-page one.
|
||||
|
||||
---
|
||||
|
||||
## 3. INGESTION WORKFLOW (TRIGGERED ON DEMAND)
|
||||
|
||||
When the user says "Ingest", "Sync the wiki", or "Update the Wiki" (for syncing this repo's own git history with its remote, see the sync-changes skill under `.claude/skills/` instead):
|
||||
|
||||
1. **Process Inbox:** Scan `raw/inbox/` for new material. Move processed items to `raw/` for archiving. If `raw/inbox/` is empty, scan `raw/` directly.
|
||||
|
||||
2. **Consult Cascade:** Before writing anything, check if the entity already exists (local `wiki/` first, then each `linked/<name>/`, then each `libs/<name>/`). The local `wiki/` always wins. Upstream content is informative but can be overridden locally.
|
||||
|
||||
3. **Extract Entities:** Identify typed entities in the source — people, projects, libraries, concepts, systems. Create entity pages in `wiki/entities/<entity-name>.md` if they don't exist. Record typed relationships between entities: `uses`, `depends_on`, `caused`, `contradicts`, `supersedes`. Store edge data in `wiki/graph/edges.json`.
|
||||
|
||||
4. **Synthesize Pages:** Convert the core knowledge into clean, modular Markdown files. Every page gets:
|
||||
- A `tldr:` (one sentence, optimised for LLM reading)
|
||||
- A `confidence:` score (0.0–1.0 based on source corroboration)
|
||||
- A `quality:` self-score (0.0–1.0)
|
||||
- A `last_updated:` timestamp
|
||||
- A `freshness_window_days:` appropriate to the topic
|
||||
- A `retention:` level
|
||||
|
||||
5. **Link & Cross-Reference:** Use **both** `[[Wikilinks]]` and standard `[markdown](path.md)` links on every cross-reference. This ensures the wiki works in Obsidian, GitHub, and CLI tools. Where useful, reference upstream files at `linked/<name>/...` or `libs/<name>/...`.
|
||||
|
||||
6. **Update Index & Log:** Add new pages to the routing table in `wiki/index.md` with a **Use when** description. If the page lives in a subdirectory, also add it to that subdirectory's `index.md`. Append a log entry to the most specific applicable log — the subdirectory's `log.md` if it has one, otherwise `wiki/log.md`. If this step creates a brand-new `wiki/<topic>/` subdirectory, immediately create that subdirectory's `index.md` per the Recursive Index & Log Convention.
|
||||
|
||||
---
|
||||
|
||||
## 4. QUERY WORKFLOW
|
||||
|
||||
When answering a question or researching a topic:
|
||||
|
||||
1. **Read the index** — `wiki/index.md` first. Match query keywords against **Use when** triggers.
|
||||
2. **Read TLDRs** — for any matched page, read its `tldr:` frontmatter first. If it answers the query, stop.
|
||||
3. **Load full pages** — only if the TLDR was insufficient.
|
||||
4. **Walk the graph** — if the entity has relationships in `wiki/graph/edges.json`, follow them to discover connected pages (e.g. "what depends on X?").
|
||||
5. **Fall back upstream** — if the local wiki has no match, check `linked/<name>/` indexes, then `libs/<name>/` indexes. Apply cascade priority throughout.
|
||||
|
||||
---
|
||||
|
||||
## 5. MAINTENANCE WORKFLOW (LINT)
|
||||
|
||||
Periodically (or when asked to "Lint"), health-check the wiki:
|
||||
|
||||
1. **Conformance check** — verify every non-reserved `.md` file under `wiki/` (i.e. excluding `index.md` and `log.md`) has parseable YAML frontmatter with a non-empty `type` field. Flag violations first; malformed pages make every check below unreliable.
|
||||
|
||||
2. **Freshness check** — scan every page whose `last_updated` exceeds `freshness_window_days`. Flag as stale; suggest the user confirm or update the content.
|
||||
|
||||
3. **Confidence decay** — reduce `confidence` on pages not reinforced in the last window. Pages below 0.3 confidence get flagged for re-review.
|
||||
|
||||
4. **Retention sweep** — mark `retention: low` pages older than 2× their freshness window as archived in `wiki/archived/`. Do not delete — move with a note in the log.
|
||||
|
||||
5. **Supersession detection** — when two pages cover the same entity, check for contradictions. If one is newer, add `supersedes` / `superseded_by` links. Preserve the old page but mark it stale.
|
||||
|
||||
6. **Orphan detection** — find pages with no inbound links. Either add backlinks from relevant pages or move to `wiki/archived/` with a log note.
|
||||
|
||||
7. **Graph consistency** — verify every edge in `wiki/graph/edges.json` points to an existing entity page. Remove or fix broken edges.
|
||||
|
||||
8. **Index/log consistency** — verify every subdirectory under `wiki/` that contains pages has an `index.md` listing all of them, and that no single change is recorded in both a subdirectory `log.md` and the root `wiki/log.md`.
|
||||
|
||||
9. **Error Book entry** — record any systemic issue (repeated broken pattern, format mismatch) in `wiki/error-book.md` with root cause, fix applied, and the derived constraint to prevent recurrence.
|
||||
|
||||
Auto-fix what you can (broken links, missing backlinks, stale flags). Report what you cannot.
|
||||
|
||||
---
|
||||
|
||||
## 6. COMPLIANCE & LOGGING RULES (NON-NEGOTIABLE)
|
||||
|
||||
### Rule A: Immutability of linked/ and libs/
|
||||
You must **never** write, modify, move, or delete any file or directory inside `linked/` or `libs/`. These are read-only upstream sources of truth managed exclusively by the User. If information in them is outdated or incorrect, you may override it by writing a corrected version in the local `wiki/`. The local version will take priority in the cascade lookup.
|
||||
|
||||
### Rule B: The Wiki Change Log (`wiki/log.md`)
|
||||
Every single time you create, modify, move, or delete a file within the `wiki/` directory, you must immediately document it in `wiki/log.md` before proceeding.
|
||||
- **Ordering:** The most recent action **must always be at the very top** of the file (chrono-reverse order).
|
||||
- **Format Per Entry:**
|
||||
```markdown
|
||||
## [YYYY-MM-DD HH:MM] - [ACTION TYPE: e.g., CREATE/UPDATE/DELETE]
|
||||
- **File Affected:** `wiki/path/to/file.md`
|
||||
- **Description:** Brief summary of what knowledge was added or altered.
|
||||
- **Source:** [e.g., Chat conversation, raw/notes.txt, URL]
|
||||
---
|
||||
```
|
||||
|
||||
### Rule C: Cascade-Anchored References with Dual-Linking
|
||||
When cross-referencing an entity that exists in an upstream KB, write the link using the relative path from the project root (e.g., `linked/<name>/wiki/concepts/foo.md` or `libs/<name>/docs/bar.md`). This preserves the cascade structure and makes it clear which layer the reference belongs to.
|
||||
|
||||
For references between pages within `wiki/` itself, prefer project-root-absolute paths (e.g. `/wiki/entities/foo.md`) over relative paths (`../entities/foo.md`). Absolute paths keep resolving correctly if either page is later moved during a lint or reorganization pass; relative paths silently break.
|
||||
|
||||
Use **both** `[[Wikilinks]]` (Obsidian-compatible) and standard `[markdown](path.md)` links on every cross-reference. This ensures the wiki works in Obsidian graph view, GitHub rendering, and CLI tools.
|
||||
|
||||
### Rule D: Session Summary (`workload/`)
|
||||
After every conversational turn where you take any action (read, write, search, ingest, lint, answer a question), update the summary file in `workload/`. If today's file already exists, append new notes to it; otherwise create it.
|
||||
- **Naming:** `workload/YYYY-MM-DD_summary.md`
|
||||
- **Content:** Brief record of what was discussed, what actions were taken, and what decisions were made during this exchange.
|
||||
- **Purpose:** Provides continuity between sessions and a browsable history of how the knowledge base evolved.
|
||||
|
||||
### Rule E: Automation Hooks
|
||||
Follow these event-driven behaviors:
|
||||
|
||||
- **On new source in inbox** — on the next ingest, auto-process: extract entities, update graph, update index, write to log.
|
||||
- **On session start** — read `wiki/index.md` and the latest `workload/` summary to load relevant context.
|
||||
- **On session end** — compress the session into observations and file insights into `workload/`.
|
||||
- **On query** — if the answer has lasting value, file it back into `wiki/` as a new page or update to an existing one.
|
||||
- **On memory write** — check for contradictions with existing wiki content. If found, apply supersession (link old → new) and log it.
|
||||
- **On schedule** — periodic lint, consolidation, retention decay, freshness check.
|
||||
|
||||
### Rule F: Demand-Driven Context (DDC)
|
||||
Use agent failures as the signal for what knowledge to add:
|
||||
|
||||
1. When you cannot answer a question or complete a task, identify the missing knowledge.
|
||||
2. Propose a minimal entity or page to fill the gap.
|
||||
3. The user approves or provides the source material.
|
||||
4. Add it to `raw/inbox/` or describe it in chat.
|
||||
5. Next ingest cycle incorporates it.
|
||||
|
||||
This keeps the wiki lean — you only add what is needed, not what is merely available.
|
||||
1
CLAUDE.md
Symbolic link
1
CLAUDE.md
Symbolic link
|
|
@ -0,0 +1 @@
|
|||
AGENTS.md
|
||||
|
|
@ -1,24 +0,0 @@
|
|||
# Session Summary — 2026-05-30
|
||||
|
||||
## Actions Taken
|
||||
- Transformed `instruction.md` from Karpathy-style LLM Wiki prompt to **Cascade Knowledge Base** concept with layered read-only upstream KBs
|
||||
- Created `wiki/index.md`, `wiki/overview.md`, `wiki/log.md` as required core pages
|
||||
- Created `AGENTS.md` and `CLAUDE.md` with full agent instructions
|
||||
- Created `README.md` with user-facing documentation
|
||||
- Researched 16 improvement ideas from web, wrote to `ideas.md`
|
||||
- Added `libs/` directory (git-managed copies, read-only, gitignored)
|
||||
- Added `outputs/` and `tmp/` directories
|
||||
- Added Index-First Navigation rule (recursive index.md scanning across all layers)
|
||||
- Implemented all 14 selected ideas: frontmatter schema (confidence, tldr, quality, supersedes, freshness, retention), entity extraction & graph, lazy-loading index with Use When triggers, TLDR-first query layer, self-healing lint, dual-linking, automation hooks, error book, DDC, inbox workflow
|
||||
- Created `wiki/graph/`, `wiki/entities/`, `raw/inbox/` directories
|
||||
- Created `wiki/error-book.md` stub
|
||||
- Restructured `wiki/index.md` into a routing table
|
||||
- Removed `ideas.md` and its reference from README
|
||||
|
||||
## Decisions Made
|
||||
- Cascade priority: wiki/ → linked/ → libs/ (alphabetical within each layer)
|
||||
- linked/ for symlinks, libs/ for git-cloned copies (both read-only, never touched by agent)
|
||||
- Local wiki always overrides upstream content
|
||||
- Frontmatter schema standardized across all wiki pages
|
||||
- Dual-linking ([[wikilinks]] + markdown) required for all cross-references
|
||||
- `tmp/`, `libs/` gitignored; `outputs/` tracked
|
||||
|
|
@ -1,41 +0,0 @@
|
|||
# Session Summary — 2026-07-13
|
||||
|
||||
## Actions Taken
|
||||
- Researched Google Cloud's Open Knowledge Format (OKF v0.1, published June 2026): a minimal markdown+YAML-frontmatter spec for portable, agent/human-consumable knowledge bundles (`type` required field, `resource` field, recursive `index.md`/`log.md`, root-absolute links, formal conformance criteria, `okf_version`).
|
||||
- Compared OKF against this repo's Cascade Knowledge Base approach and identified 8 areas where OKF's design is stronger (typing, link stability, recursive progressive disclosure, hierarchical logs, resource linking, conformance checking, schema versioning, generic interop tooling), while noting this repo's cascade layering, confidence/quality/retention decay, and session continuity go beyond what OKF specifies.
|
||||
- Implemented propositions 1–7 (OKF-inspired improvements), applied identically to `CLAUDE.md` and `AGENTS.md`:
|
||||
1. Added required `type` frontmatter field + optional `resource` field to the Page Frontmatter Schema.
|
||||
2. Added `resource` field (see above, same schema change).
|
||||
3. Added guidance to Rule C to prefer project-root-absolute links (`/wiki/entities/foo.md`) for intra-wiki cross-references over relative paths.
|
||||
4. Added a "Recursive Index & Log Convention" subsection under Index-First Navigation; created `wiki/entities/index.md` and `wiki/graph/index.md` as the first real subdirectory indexes; updated the Ingestion Workflow (step 6) to create subdirectory indexes/logs going forward.
|
||||
5. Made `wiki/log.md` an explicit root-level rollup, with subdirectories permitted their own `log.md` once they have independent history — avoids one unbounded flat log file.
|
||||
6. Added a "Conformance check" as lint step 1 (verify parseable frontmatter + non-empty `type` on every non-reserved page) and an "Index/log consistency" check as lint step 8.
|
||||
7. Added a `kb_schema_version` field (bundle-level, on `wiki/index.md` only) with minor/major bump rules, documented in a new "Schema Versioning" subsection.
|
||||
- Propagated the same documentation updates to `wiki/index.md`, `wiki/overview.md`, and `README.md` for consistency, and logged each wiki-level change in `wiki/log.md`.
|
||||
|
||||
## Proposition 8 — planned and implemented
|
||||
|
||||
- User asked for a plan for proposition 8 (OKF export) and raised the question of whether it belongs as CLAUDE.md/AGENTS.md prose or as a command/skill. Entered plan mode, delegated design validation to a Plan agent, then wrote and got approval for a plan at `/home/mkopec/.claude/plans/twinkly-wobbling-harbor.md`.
|
||||
- **Decision:** implemented as a project-scoped Claude Code Skill (`.claude/skills/export-okf/SKILL.md`), not new CLAUDE.md/AGENTS.md prose. Rationale: OKF export is a deterministic, occasionally-invoked transformation with a large fixed mapping ruleset — exactly what Skills' lazy-loading is for, versus permanently taxing every session's context via the always-loaded instruction files. Project-scoped (not `~/.claude/skills/`) because the mapping is coupled to this repo's exact schema and travels with it in the same commits.
|
||||
- Closed a discoverability gap for non-Claude-Code agents: added one generic sentence to `CLAUDE.md`/`AGENTS.md` §1 (`outputs/` line) pointing at `.claude/skills/` in general, without naming this specific skill, so the pointer never needs updating as skills are added/removed.
|
||||
- Fixed a prerequisite gap discovered during planning: `wiki/overview.md` and `wiki/error-book.md` had zero YAML frontmatter and would have failed the conformance lint check added earlier this session. Retrofitted both with `type`/`tldr`/`last_updated`.
|
||||
- Updated `README.md` with a "OKF Export" feature bullet and a note clarifying `outputs/okf/` is a fully-regenerated build artifact that stays git-tracked (consistent with the existing `outputs/` convention) but is never auto-committed by the skill itself.
|
||||
- Wrote the full `export-okf` SKILL.md: 8-step procedure (read source → clear+rebuild outputs/okf/ → transform concept-doc frontmatter/links → regenerate index.md files → regenerate log.md files → handle non-reserved special pages → validate output conformance → report), plus edge cases (empty entities/graph dirs, missing `type`, future `archived/`, determinism on re-run).
|
||||
- Have not yet run the skill end-to-end (wiki/entities and wiki/graph are still empty, so there's little to export yet) — first real exercise of the skill will happen on the next ingest that populates entities.
|
||||
|
||||
## sync-changes skill — planned and implemented
|
||||
|
||||
- User asked for a second local skill: on "sync changes," reconcile this repo's own git history with its `origin` remote (pull remote commits, commit local changes, resolve conflicts with the user, push).
|
||||
- Clarified scope up front via AskUserQuestion: "git" means this repo's own `origin` remote (`https://codeberg.org/Valdec/llm-wiki-cascade.git`), not `libs/`; and unlike `export-okf`, auto-push is explicitly approved here — the skill commits and pushes on its own once conflicts are resolved.
|
||||
- Checked real repo state (read-only): `origin` is configured, but local `main` has **zero commits** — everything was untracked at the time of this session. This made the "local has no commits, remote may already have history" first-run case a live scenario, not a hypothetical, so it's handled as an explicit hard-stop path (ask the user to choose merge / remote-wins / stop) rather than an automatic guess.
|
||||
- Entered plan mode again given the risk of automated git push; delegated design validation to a Plan agent (git command sequencing, conflict-presentation mechanics, first-run handling), then resolved two remaining open questions via AskUserQuestion: (1) never offer a force-push/discard-remote option even in the first-run menu — recommended and confirmed; (2) reword the existing bare `"Sync"` Ingestion trigger to `"Sync the wiki"` to reduce ambiguity with the new skill's `"sync changes"` trigger — recommended and confirmed.
|
||||
- Implemented: `.claude/skills/sync-changes/SKILL.md` (5-step procedure: pre-flight safety checks incl. secrets scan → detect first-run/unrelated-histories case → steady-state commit-then-fetch-merge-then-push flow → present each conflict block via AskUserQuestion with keep-local/keep-remote/provide-merged-text options → structured report), plus edge cases (nothing-to-sync, push-rejected-twice, no-remote-configured).
|
||||
- Updated `CLAUDE.md`/`AGENTS.md` §3 trigger wording (`"Sync"` → `"Sync the wiki"` + pointer to the new skill) and added a "Git Sync (on demand)" bullet to `README.md`, matching the `export-okf` precedent.
|
||||
- Have not yet run the skill for real — it would actually commit and push this session's changes to the live Codeberg remote, which is a real external action, so it's left for the user to explicitly trigger with "sync changes" rather than auto-run as part of this session's "verification."
|
||||
|
||||
## Decisions Made
|
||||
- Keep `CLAUDE.md` and `AGENTS.md` byte-identical; every instruction edit is applied to both in lockstep.
|
||||
- `type` is required per OKF precedent, but the rest of the existing schema (confidence/quality/retention/etc.) is retained as-is — OKF's minimalism is adopted piecemeal, not wholesale, since the decay/retention/cascade machinery has no OKF equivalent and is worth keeping.
|
||||
- `kb_schema_version` lives only on `wiki/index.md` (bundle-level), not on every page, matching OKF's `okf_version` convention — and is dropped (not relocated) when exporting to OKF, since OKF's root index.md frontmatter is spec-limited to `okf_version` only.
|
||||
- OKF export lives in `.claude/skills/export-okf/`, project-scoped rather than global, so it stays co-versioned with this repo's schema.
|
||||
- Git-level sync lives in `.claude/skills/sync-changes/`, project-scoped for the same reason, and is explicitly authorized to auto-commit/auto-push (unlike export-okf) per direct user confirmation — but never force-pushes and always stops to ask on the first-run unrelated-histories case.
|
||||
Loading…
Add table
Reference in a new issue