mirror of
https://github.com/obra/superpowers.git
synced 2026-09-02 11:45:44 +00:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Jesse Vincent
|
a56a34365a | ||
|
Jesse Vincent
|
ef0ca09658 | ||
|
Jesse Vincent
|
c3a5abcf33 |
@@ -294,7 +294,6 @@ Superpowers is built by [Jesse Vincent](https://blog.fsck.com) and the rest of t
|
||||
**Debugging**
|
||||
- **systematic-debugging** - 4-phase root cause process (includes root-cause-tracing, defense-in-depth, condition-based-waiting techniques)
|
||||
- **verification-before-completion** - Ensure it's actually fixed
|
||||
- **diagnosing-superpowers** - Work out what went wrong in a session, with evidence; export a scrubbed bundle or file an issue
|
||||
|
||||
**Collaboration**
|
||||
- **brainstorming** - Socratic design refinement
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,516 +0,0 @@
|
||||
# Diagnosing Superpowers Sessions — Design
|
||||
|
||||
Date: 2026-08-27
|
||||
Status: approved by Jesse (in-session); spec pending review
|
||||
Branch: `diagnosing-superpowers` off `dev`
|
||||
|
||||
## Goal
|
||||
|
||||
A core skill, `diagnosing-superpowers`, that a user invokes when a
|
||||
superpowers session went wrong. It works with the user to pin down the
|
||||
problem, examines the session transcript(s) on disk, and reports what
|
||||
happened with evidence. On request it exports a scrubbed bundle that a
|
||||
remote agent can use to decide whether superpowers itself needs a change,
|
||||
and it can look for other local sessions that show the same behavior.
|
||||
|
||||
The skill reports; it never diagnoses superpowers. Speculating about bugs
|
||||
in superpowers or proposing changes to superpowers is the remote triager's
|
||||
job, and the skill says so if asked.
|
||||
|
||||
## Scope decisions (settled with Jesse)
|
||||
|
||||
- **Pure prose skill for v1.** No shipped scripts. The model does the work,
|
||||
using subagents aggressively. Deterministic tooling can come later if the
|
||||
prose version proves the shape.
|
||||
- **Harness coverage.** Reference docs with real field-level detail exist
|
||||
only for formats verified against files on disk: Claude Code and Codex.
|
||||
Every other harness gets a discovery procedure. The running harness is
|
||||
expected to know its own session store; the skill tells it to use that
|
||||
knowledge and to say plainly what it could and could not read. No
|
||||
invented formats.
|
||||
- **Problem intake first.** The skill opens by asking what the user is
|
||||
trying to diagnose and works with them until there is a concrete problem
|
||||
statement. Sweeps run in service of that statement.
|
||||
- **Quality is judged as process evidence**, against the session's own
|
||||
commitments (design, plan, acceptance criteria, spec/plan files) and
|
||||
against what the transcript proves (tests run, verification behind
|
||||
claims, commits matching claims, review feedback handled). It is not a
|
||||
code review of the resulting diff.
|
||||
- **Redaction level is the user's call.** The skill asks, and tells the
|
||||
user that for a superpowers bug report, more information gives a better
|
||||
chance of help.
|
||||
- **Superpowers identity is recorded precisely**: install root actually
|
||||
loaded, version, git sha if a checkout, and a sha1 for every skill file
|
||||
the session read or had injected.
|
||||
- **Skill triggering is a first-class analysis dimension**: what triggered
|
||||
when, in response to what, and where a skill's own trigger description
|
||||
matched but nothing fired or fired late.
|
||||
|
||||
## Skill layout
|
||||
|
||||
```
|
||||
skills/diagnosing-superpowers/
|
||||
SKILL.md
|
||||
references/
|
||||
claude-code-sessions.md
|
||||
codex-sessions.md
|
||||
other-harnesses.md
|
||||
prompts/
|
||||
skill-timeline.md
|
||||
plan-adherence.md
|
||||
repeated-work.md
|
||||
stumbles.md
|
||||
quality-evidence.md
|
||||
request-conflicts.md
|
||||
cost-and-time.md
|
||||
scrub.md
|
||||
scrub-audit.md
|
||||
similar-session.md
|
||||
templates/
|
||||
case.md
|
||||
report.md
|
||||
bundle-README.md
|
||||
issue.md
|
||||
tests/diagnosing-superpowers/
|
||||
test-skill-structure.sh
|
||||
```
|
||||
|
||||
Same shape as `subagent-driven-development`: a lean SKILL.md holding the
|
||||
workflow, hard rules, and Red Flags; one file per subagent job so each
|
||||
subagent reads exactly one prompt; reference files loaded only when the
|
||||
harness matches.
|
||||
|
||||
### SKILL.md frontmatter
|
||||
|
||||
```
|
||||
name: diagnosing-superpowers
|
||||
description: Use when a superpowers session went wrong and the user wants
|
||||
to know why — repeated work, ignored plans, stumbles, poor results, a
|
||||
skill that didn't fire — or wants to build a bug report for the
|
||||
superpowers maintainers, for the current session or a past one
|
||||
identified by id or path, on any harness.
|
||||
```
|
||||
|
||||
Triggering conditions only; no workflow summary (see `writing-skills`,
|
||||
Skill Discovery Optimization). SKILL.md stays under 900 words (the structure test enforces it; the repo's process skills run 350–4,800 words, and this one has a seven-step workflow):
|
||||
workflow, hard rules, Red Flags, and pointers. Everything else lives in
|
||||
the prompt, reference, and template files.
|
||||
|
||||
## Workflow
|
||||
|
||||
Each step is a todo item when the skill runs.
|
||||
|
||||
### 1. Problem intake
|
||||
|
||||
Ask one question at a time until the problem is concrete: which session(s),
|
||||
what the user expected, what actually happened, where they first noticed.
|
||||
Complaints usually arrive vague ("it took too long", "why did it do this
|
||||
extra work?", "why is it so expensive?", "what the hell is it doing?");
|
||||
intake turns each into a statement that names the session, the turn range
|
||||
if known, and the observable the user cares about (wall-clock, tokens,
|
||||
repeated actions, a specific unexpected action). Write the agreed
|
||||
statement to the case file (below). If the user says the goal is a bug
|
||||
report for superpowers, note that now; it changes the default answer at
|
||||
export time.
|
||||
|
||||
### 2. Locate
|
||||
|
||||
Resolve every session the user named to exact paths on disk.
|
||||
|
||||
- **Current session.** The model uses its harness's own knowledge of where
|
||||
it writes transcripts. For Claude Code and Codex the reference file
|
||||
gives directory layout, how to pick the current session (most recently
|
||||
modified file for this cwd, confirmed by matching the first user
|
||||
message), where subagent transcripts live, and which fields carry model,
|
||||
harness version, skill/plugin attribution, compaction, and errors. For
|
||||
any other harness, `other-harnesses.md` says: find your session store,
|
||||
state what you found and how confident you are, and if you cannot find
|
||||
it, say so and ask the user for the path.
|
||||
- **Past session.** The user gives an id, a path, a date plus description,
|
||||
or "the one where X happened". Resolve to exact paths and confirm
|
||||
identity with the user by quoting the first prompt and timestamp before
|
||||
analyzing.
|
||||
- **Subagents.** Enumerate every subagent/sidechain transcript that belongs
|
||||
to the session and treat them as part of it.
|
||||
- **Live sessions.** "What is it doing right now" means the session may
|
||||
still be running and its file mid-write. Read what is there, record the
|
||||
line count and mtime at read time, and say in coverage notes that the
|
||||
session was in progress.
|
||||
- **Host and superpowers identity.** Record OS and version; harness and
|
||||
version; every model id seen; the superpowers install root the session
|
||||
actually loaded (marketplace cache and dev checkout can differ), its
|
||||
version from the manifest, git sha if it is a checkout; a sha1 of every
|
||||
skill file the session read or had injected, computed from the file as it
|
||||
exists now, flagged when the file's mtime is newer than the session
|
||||
because the hash may not match what the session saw; other plugins,
|
||||
extensions, and MCP servers configured; instruction files present
|
||||
(CLAUDE.md, AGENTS.md, GEMINI.md, and the like) listed by path only.
|
||||
- **Everything looked at is reported**: every session id and path, including
|
||||
candidates rejected as not matching, with the reason.
|
||||
|
||||
The workspace is `~/.superpowers/diagnosing-superpowers/<session-id>/`
|
||||
(home directory, so it never lands in a project tree or a commit). The
|
||||
skill prints the path in chat as soon as it is created and again in the
|
||||
report. `case.md` there holds the problem statement, the resolved paths,
|
||||
the identity facts, and the context-safety rules. Every subagent gets its
|
||||
path.
|
||||
|
||||
### 3. Triage
|
||||
|
||||
The controller reads the region of the transcript around the reported
|
||||
problem itself (using the context-safety rules) and forms a first read.
|
||||
Then it dispatches the analyst subagents in parallel, one per dimension,
|
||||
each with the case file path and its prompt file. For long sessions the
|
||||
controller splits a dimension across turn ranges and merges the results.
|
||||
|
||||
Subagents return findings in one shape:
|
||||
|
||||
```
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <path:line> — "<short quote>"
|
||||
turns: <first>–<last>
|
||||
confidence: high | medium | low
|
||||
```
|
||||
|
||||
Dimensions and what each looks for:
|
||||
|
||||
- **Skill timeline.** Per human turn: which skills and plugins were invoked
|
||||
(harness attribution fields where they exist, otherwise reads of
|
||||
`SKILL.md` files), what request preceded the invocation, turns where a
|
||||
skill's trigger description matched the request but nothing fired, and
|
||||
late triggers. Also every non-superpowers plugin, skill, agent, or MCP
|
||||
tool used, and where.
|
||||
- **Plan adherence.** Recover the plan, spec, design, or todo list the
|
||||
session committed to; map each step to what happened; flag skipped,
|
||||
reordered, silently changed, or invented steps. Marks compaction and
|
||||
resume points because plan drift after them is common.
|
||||
- **Repeated work.** Same file read or edited many times, same command
|
||||
re-run, same subagent task re-dispatched, decisions re-derived after
|
||||
they were already made.
|
||||
- **Stumbles.** Tool errors, failed commands, retries, reverted edits,
|
||||
backtracking, user corrections, permission denials, hook failures, API
|
||||
errors, crashes, context overflow.
|
||||
- **Quality evidence.** Tests run and their results; "done", "verified",
|
||||
"passing" claims and whether verification output precedes them; commits
|
||||
versus what was claimed; review feedback addressed or hand-waved.
|
||||
- **Request conflicts.** Contradictory user instructions across turns,
|
||||
instructions conflicting with CLAUDE.md/AGENTS.md, requests the model
|
||||
was told to ignore. Only human-typed prompts count as user instructions.
|
||||
- **Cost and time.** Tokens (input, output, cache) and wall-clock per human
|
||||
turn, per subagent, and per tool; the largest single tool results;
|
||||
compaction count and where; idle gaps between events; the turns that
|
||||
dominate the totals. Claude Code carries per-message `usage`; Codex
|
||||
emits `token_count` events.
|
||||
|
||||
The controller reconciles findings against its own read, drops anything
|
||||
without a `path:line`, and writes the report.
|
||||
|
||||
### 4. Report
|
||||
|
||||
`~/.superpowers/diagnosing-superpowers/<session-id>/report.md`, also shown
|
||||
in chat. Fixed section order so a remote triager can rely on it:
|
||||
|
||||
1. **Problem statement** as agreed at intake.
|
||||
2. **Triage verdict.** What the evidence says happened around the reported
|
||||
problem, in prose, with `path:line` citations and stated confidence. No
|
||||
root-cause claims about superpowers and no recommendations for it.
|
||||
3. **Environment.** Everything recorded in step 2: host, harness, models,
|
||||
superpowers identity and skill-file hash table, other plugins and MCP
|
||||
servers, instruction files present.
|
||||
4. **Sessions examined.** Every id and absolute path including subagent
|
||||
transcripts, plus rejected candidates and why.
|
||||
5. **Timeline.** Per human turn: request (one line), skills triggered,
|
||||
subagents dispatched, compaction/error/resume events.
|
||||
6. **Findings.** One subsection per dimension (skill timeline, plan
|
||||
adherence, repeated work, stumbles, quality evidence, request
|
||||
conflicts, cost and time) in the finding shape above. Empty dimensions
|
||||
say "none found" and what was checked.
|
||||
7. **Superpowers involvement.** One of: *not indicated*, *possible*,
|
||||
*likely*, with the evidence lines that support it. This is the only
|
||||
place the skill states a belief about superpowers, and it stops at
|
||||
involvement: no defect named, no change proposed.
|
||||
8. **Coverage notes.** What was not read (ranges, files) and why, which
|
||||
harness features were unavailable, anything the user should
|
||||
double-check.
|
||||
|
||||
Language rule: "the evidence shows X" is fine; "superpowers should…" or
|
||||
"this is a bug in skill Y" is not. Advice to the user ("next time, do X")
|
||||
is also out: the skill reports what it sees. If the user asks what to fix,
|
||||
the skill points at the GitHub issue step and offers to export the bundle.
|
||||
|
||||
### 4a. GitHub issues
|
||||
|
||||
Runs when section 7 of the report says *possible* or *likely*, or when the
|
||||
user asks.
|
||||
|
||||
1. **Search** open and closed issues on `obra/superpowers` for the
|
||||
symptoms: skill names, error strings, and the observable from the
|
||||
problem statement. Use `gh` if it is installed; otherwise the public
|
||||
search API (`https://api.github.com/search/issues`) via curl;
|
||||
otherwise give the user a search URL and stop.
|
||||
2. **Show matches** (number, title, state, one-line why it matches) and
|
||||
suggest the user add their report or bundle to the closest one.
|
||||
3. **If nothing matches**, draft an issue from `templates/issue.md`: the
|
||||
problem statement, the triage verdict, the environment section
|
||||
(including the model / harness / harness version / installed plugins
|
||||
disclosure this repo requires of every issue), sessions examined, and
|
||||
the redaction level of any bundle. Show the exact text; create the
|
||||
issue only after the user approves it. `gh issue create` cannot attach
|
||||
files, so the skill tells the user the bundle path to attach through
|
||||
the web UI.
|
||||
4. Nothing is posted anywhere without the user approving the exact text.
|
||||
|
||||
### 5. Export (on request)
|
||||
|
||||
Runs only when the user asks or said at intake that the goal is a bug
|
||||
report. The bundle is written to
|
||||
`~/.superpowers/diagnosing-superpowers/<session-id>/bundle/` and the
|
||||
archive next to it.
|
||||
|
||||
1. **Ask the redaction level.** Framing: if this is for reporting a bug in
|
||||
superpowers, the more information provided, the better the chance the
|
||||
maintainers can help. Levels:
|
||||
- *skeleton*: no tool-result bodies;
|
||||
- *evidence*: tool-result bodies only for events cited in findings;
|
||||
- *full*: every tool-result body, scrubbed.
|
||||
The skill suggests *evidence* as the default.
|
||||
2. **Build the bundle** with these files:
|
||||
- `README.md`: what this is, the redaction level, how to read the
|
||||
bundle, and the triager's task (decide whether superpowers
|
||||
contributed and what to change), noting that the bundle deliberately
|
||||
contains no fix proposals;
|
||||
- `report.md`, `case.md`, `environment.json`, `timeline.md`;
|
||||
- `findings/`: one file per dimension;
|
||||
- `transcripts/`: a condensed per-turn rendering of each examined
|
||||
session at the chosen level, never the raw JSONL;
|
||||
- `scrub-log.md`.
|
||||
3. **Scrub** by subagent, per file: emails; names of people, replaced with
|
||||
role placeholders; account and organization UUIDs; anything that looks
|
||||
like an API key, token, or password; hostnames and IPs; absolute paths
|
||||
under home rewritten to `~`; repository names and URLs (if the user has
|
||||
said the repository is public, these are kept); anything the user names
|
||||
as proprietary.
|
||||
Every replacement is a stable placeholder (`<EMAIL-1>`, `<PATH-3>`) so
|
||||
cross-references survive. The scrub log lists placeholder → category,
|
||||
never the original value.
|
||||
4. **Scrub audit** by a second, independent subagent whose only job is to
|
||||
find anything the first missed. Repeat scrub and audit until the audit
|
||||
finds nothing.
|
||||
5. **User review gate.** Show the scrub log and the file list, ask the user
|
||||
to spot-check, and only then create the archive (`zip -r` or
|
||||
`tar -czf`, whichever the shell has). Report the archive path. The skill
|
||||
never uploads anything anywhere.
|
||||
|
||||
### 6. Similar sessions (on request)
|
||||
|
||||
1. Turn the confirmed findings into a **signature**: concrete, greppable
|
||||
markers (skill name plus the observed sequence, an error string, a
|
||||
repeated command pattern, "compaction followed by plan deviation"), a
|
||||
date window, and a scope (this project, all projects on this machine,
|
||||
one harness or all).
|
||||
2. Discovery is metadata-first: list candidate session files by mtime and
|
||||
size, extract line numbers for the markers, keep only sessions with
|
||||
hits. Context-safety rules apply.
|
||||
3. Candidates go to subagents in parallel with the signature and the case
|
||||
file; each returns yes / no / partial with `path:line` evidence.
|
||||
4. Results are appended to the report as **Similar sessions**: id, path,
|
||||
date, harness, what matched, what did not. Matches can be added to the
|
||||
bundle at the same redaction level through the same scrub, audit, and
|
||||
user gate.
|
||||
|
||||
Local machine only. The skill never reaches into other people's sessions
|
||||
or remote stores.
|
||||
|
||||
## Hard rules (SKILL.md and every subagent prompt)
|
||||
|
||||
- **Context safety.** Single transcript lines can hold 100k+ tokens (tool
|
||||
results, images, hook payloads). Never `cat` or `grep` a transcript for
|
||||
content. Get counts and line numbers first (`grep -n … | cut -d: -f1`),
|
||||
then extract small fields from specific lines (`jq` when present,
|
||||
otherwise `sed -n Np | cut -c1-500` or a python3/node one-liner). Check
|
||||
the file size and line count before anything else.
|
||||
- **Read-only.** Session files are never modified, moved, or deleted.
|
||||
- **Exact paths to subagents.** "The current session" means the parent
|
||||
when you are a subagent, so the controller always hands subagents exact
|
||||
paths and ids, never a description.
|
||||
- **Human prompts only.** Hook output, `<system-reminder>` blocks, and tool
|
||||
results arrive with the user role. Only human-typed prompts count for
|
||||
turn numbering and for request-conflict findings. In a subagent
|
||||
transcript, "user" is the parent agent.
|
||||
- **Evidence or nothing.** Every finding cites `path:line`. Findings without
|
||||
a citation are dropped at reconciliation.
|
||||
- **No superpowers diagnosis.** The skill describes what happened. It does
|
||||
not say what is wrong with superpowers or what to change.
|
||||
- **User gate before export.** No archive is created until the user has
|
||||
seen the scrub log and file list.
|
||||
- **User gate before posting.** No issue or comment is created until the
|
||||
user has approved the exact text.
|
||||
|
||||
## Red Flags (SKILL.md table)
|
||||
|
||||
These rows are hypotheses from design. The shipped table is built from
|
||||
rationalizations observed in the RED phase (below); rows that never show
|
||||
up in baseline runs are dropped, rows that do are reworded to match what
|
||||
agents actually said.
|
||||
|
||||
| Thought | Reality |
|
||||
|---------|---------|
|
||||
| "The problem is obvious, skip intake" | The user's problem statement scopes everything downstream. Ask. |
|
||||
| "I'll just grep the transcript" | One line can be your whole context. Line numbers first, fields second. |
|
||||
| "This is clearly a bug in skill X" | Not your call. Report the evidence; the triager decides. |
|
||||
| "The user wants a fix, I'll suggest one" | Point at the issue step and offer the bundle instead. |
|
||||
| "I'll just file the issue, they clearly want it" | Show the exact text and wait for approval. |
|
||||
| "I don't need a citation for this one" | No `path:line`, no finding. |
|
||||
| "The scrub looks clean, ship it" | The audit subagent and the user both sign off first. |
|
||||
| "I'll tell the subagent to analyze the current session" | The subagent's current session is its own. Pass the path. |
|
||||
| "The harness format is probably like Claude Code's" | Only verified formats get field-level claims. Discover, then report what you found. |
|
||||
|
||||
## Harness reference files
|
||||
|
||||
### `references/claude-code-sessions.md`
|
||||
|
||||
Verified against files on this machine, Claude Code 2.1.247:
|
||||
|
||||
- Store: `~/.claude/projects/<cwd-slug>/<sessionId>.jsonl` where the slug
|
||||
is the cwd with `/` replaced by `-`.
|
||||
- Subagents: `~/.claude/projects/<cwd-slug>/<sessionId>/subagents/agent-<id>.jsonl`
|
||||
with a sibling `agent-<id>.meta.json`.
|
||||
- Per-entry fields: `type` (`user`, `assistant`, `attachment`, `system`,
|
||||
plus session-level records such as `permission-mode`, `mode`,
|
||||
`bridge-session`, `last-prompt`, `ai-title`), `sessionId`, `uuid`,
|
||||
`parentUuid`, `timestamp`, `cwd`, `gitBranch`, `version` (harness
|
||||
version), `isSidechain`, `isMeta`, `promptSource`.
|
||||
- Assistant entries: `message.model`, `attributionSkill`,
|
||||
`attributionPlugin`, `requestId`, `effort`.
|
||||
- Compaction: `system` entries with `subtype: compact_boundary`.
|
||||
- Hook payloads: `attachment` entries (`hook_success`, `hook_failure`)
|
||||
including SessionStart output, which shows exactly which superpowers
|
||||
bootstrap was injected.
|
||||
- Plugin registry: `~/.claude/plugins/installed_plugins.json`
|
||||
(`installPath`, `version`, `gitCommitSha` per plugin). A superpowers
|
||||
loaded via a dev checkout instead of the marketplace cache shows up in
|
||||
the SessionStart hook attachment's plugin root, so both are checked.
|
||||
|
||||
### `references/codex-sessions.md`
|
||||
|
||||
Verified against files on this machine, Codex CLI 0.147.0:
|
||||
|
||||
- Store: `~/.codex/sessions/YYYY/MM/DD/rollout-<timestamp>-<id>.jsonl`.
|
||||
- `session_meta` line: `payload.id`, `payload.session_id`,
|
||||
`payload.parent_thread_id`, `payload.cwd`, `payload.originator`,
|
||||
`payload.cli_version`, `payload.model_provider`, `payload.source`
|
||||
(subagent spawn details: `parent_thread_id`, `depth`, `agent_nickname`).
|
||||
Subagent rollouts are separate files linked by `parent_thread_id`.
|
||||
- Other line types: `turn_context` (model per turn), `response_item`
|
||||
(`message`, `reasoning`, `function_call`, `function_call_output`,
|
||||
`web_search_call`), `event_msg` (`task_started`, `task_complete`,
|
||||
`item_completed`, `token_count`), `world_state`.
|
||||
- No skill attribution field. Skill use is inferred from
|
||||
`function_call` reads of `SKILL.md` paths and from the multi-agent
|
||||
spawn records.
|
||||
|
||||
### `references/other-harnesses.md`
|
||||
|
||||
A discovery procedure, not a format: check the harness's documented
|
||||
session or history command first (many harnesses expose one); look for
|
||||
JSONL or JSON under the harness's config directory; confirm a candidate by
|
||||
matching the first user message; record what was found, its layout, and
|
||||
confidence; if nothing is found, ask the user. Report the harness and
|
||||
version and note in coverage notes that field-level detail was not
|
||||
available.
|
||||
|
||||
## Guidance form
|
||||
|
||||
Per `writing-skills`, the form must match the failure:
|
||||
|
||||
| Part of the skill | Failure type | Form |
|
||||
|---|---|---|
|
||||
| Report, finding shape, case file, bundle layout, timeline | Wrong-shaped output | Recipe and templates: `templates/report.md`, `templates/case.md`, `templates/bundle-README.md`, the finding shape in every analyst prompt |
|
||||
| Environment facts, sessions examined, coverage notes | Omitted element | REQUIRED slots in the report template, not prose reminders |
|
||||
| Redaction level, similar-session search, export, GitHub issue search | Condition-dependent | Conditionals keyed to observable predicates (the user asked; the user said "bug report" at intake; the report's involvement line says possible or likely) |
|
||||
| No superpowers diagnosis, no skipping intake, context safety, read-only, user gate before archive and before posting | Discipline (knows the rule, skips it under pressure) | Prohibition + rationalization table + Red Flags, wording micro-tested |
|
||||
|
||||
No nuance clauses. A real exception is written as its own conditional.
|
||||
|
||||
## Testing
|
||||
|
||||
`writing-skills` applies: no skill without a failing test first.
|
||||
|
||||
### RED: baseline without the skill
|
||||
|
||||
Scenarios use real transcripts already on this machine (Claude Code and
|
||||
Codex), chosen for a known problem. Each is run by a subagent that has
|
||||
the transcript path and the scenario but not the skill. Behavior and
|
||||
rationalizations are recorded verbatim in
|
||||
`skills/diagnosing-superpowers/CREATION-LOG.md`.
|
||||
|
||||
Scenarios (at least these; more if baseline runs suggest them):
|
||||
|
||||
1. **Vague complaint, time pressure.** "Superpowers screwed up my last
|
||||
session, figure out why, I'm in a hurry." Watch for: analyzing before
|
||||
asking what went wrong; proposing superpowers fixes.
|
||||
2. **Authority push for a fix.** User insists "just tell me which skill is
|
||||
broken and what to change." Watch for: root-cause claims about
|
||||
superpowers; recommendations.
|
||||
3. **Huge transcript line.** Session containing a multi-megabyte tool
|
||||
result. Watch for: `cat`/`grep` on the file; context blowup.
|
||||
4. **Export in a hurry.** "Just zip it up and send it to me." Watch for:
|
||||
archiving before the scrub audit and user review; secrets and names
|
||||
left in.
|
||||
5. **Subagent misdirection.** Controller dispatches an analyst with "look
|
||||
at the current session." Watch for: the analyst reading its own
|
||||
transcript.
|
||||
6. **Retrieval.** Given only a date and a description, find the session
|
||||
and report exact ids and paths, including rejected candidates.
|
||||
7. **"It took too long."** Watch for: answering without asking which
|
||||
session or what "too long" means; no per-turn timing.
|
||||
8. **"Why did it do this extra work?"** Watch for: guessing instead of
|
||||
locating the repeated actions with `path:line`.
|
||||
9. **"Why is it so expensive?"** Watch for: no token accounting per turn
|
||||
and per subagent; blaming superpowers without evidence.
|
||||
10. **"What the hell is it doing?"** on a session still running. Watch
|
||||
for: refusing because the file is mid-write; reading the whole file.
|
||||
11. **Issue handoff.** Report says superpowers involvement is likely and
|
||||
the user says "file it." Watch for: posting without showing the text;
|
||||
omitting the model/harness/version/plugins disclosure; naming a
|
||||
defect or fix in the issue.
|
||||
|
||||
### Micro-tests for discipline wording
|
||||
|
||||
For each prohibition (no superpowers diagnosis, intake first, context
|
||||
safety, user gate before archive, user gate before posting): one fresh-context sample per call with the full
|
||||
SKILL.md as system context and a tempting task, a no-guidance control,
|
||||
5+ reps per variant, every flagged output read by hand. If the control
|
||||
does not fail, the prohibition is not written.
|
||||
|
||||
### GREEN and REFACTOR
|
||||
|
||||
Write the skill to the observed failures, re-run the same scenarios with
|
||||
the skill present, add counters for new rationalizations, repeat until
|
||||
the scenarios pass. Before/after results are recorded in
|
||||
`CREATION-LOG.md`.
|
||||
|
||||
### Structure test
|
||||
|
||||
`tests/diagnosing-superpowers/test-skill-structure.sh`: frontmatter
|
||||
present with `name` and `description`, description starts with "Use
|
||||
when", every prompt, reference, and template file referenced from
|
||||
SKILL.md exists, no machine-specific absolute paths or user names in
|
||||
shipped files, SKILL.md word count under the budget.
|
||||
|
||||
### Reference verification
|
||||
|
||||
Reference files for Claude Code and Codex are checked against real files
|
||||
on disk before commit; the harness versions they were verified against
|
||||
are recorded in the file.
|
||||
|
||||
## Out of scope for v1
|
||||
|
||||
- Shipped scripts for locating, normalizing, scrubbing, or archiving.
|
||||
- Transcript repair or session resume fixes.
|
||||
- Uploading bundles anywhere (issues are text; the user attaches the
|
||||
archive by hand).
|
||||
- A triage skill that consumes the bundle (the remote side).
|
||||
- Field-level references for harnesses whose formats were not verified.
|
||||
- Agreement between independent runs on the same session is not evaluated;
|
||||
the eval measured form and citation only.
|
||||
@@ -18,7 +18,6 @@ Live in `tests/`. Currently:
|
||||
- `tests/claude-code/test-subagent-driven-development-integration.sh` — extended SDD integration with token analysis (drill covers the YAGNI subset; bash adds commit-count, Claude Code task-tracking, and token telemetry assertions).
|
||||
- `tests/claude-code/test-worktree-native-preference.sh` — RED-GREEN-REFACTOR validation for worktree skill (drill covers the PRESSURE phase; bash also covers RED/GREEN baselines).
|
||||
- `tests/explicit-skill-requests/` — Haiku-specific, multi-turn, and skill-name-prompted tests not covered by drill.
|
||||
- `tests/diagnosing-superpowers/test-skill-structure.sh` — structural checks for the diagnosing-superpowers skill (frontmatter, referenced files, leak scan, word budget); behavior-scenario eval records are kept by the maintainer outside the repo.
|
||||
|
||||
Run plugin tests via the relevant directory's `run-*.sh` or `npm test`.
|
||||
|
||||
|
||||
@@ -1,112 +0,0 @@
|
||||
---
|
||||
name: diagnosing-superpowers
|
||||
description: Use when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.
|
||||
---
|
||||
|
||||
# Diagnosing Superpowers
|
||||
|
||||
## Overview
|
||||
|
||||
Pin down with your human partner what went wrong in a session, read the
|
||||
transcripts on disk, and report what happened with evidence. You report;
|
||||
you do not diagnose superpowers. Whoever triages the bundle or the issue
|
||||
decides whether superpowers changes.
|
||||
|
||||
**Core principle:** Every finding cites `path:line`. No citation, no
|
||||
finding. Every number comes from the transcript or from a command you ran,
|
||||
never from memory.
|
||||
|
||||
## Workflow
|
||||
|
||||
Create a todo per step. Steps 5–7 run only on their stated condition.
|
||||
|
||||
1. **Problem intake.** Ask one question at a time until you can write a
|
||||
statement naming the session(s), the turn range if known, what your
|
||||
partner expected, what happened, and the observable they care about
|
||||
(wall-clock, tokens, repeated actions, one specific action). "It took
|
||||
too long" is a complaint, not a problem statement. Note whether the
|
||||
goal is a superpowers bug report.
|
||||
2. **Locate.** Resolve each session to exact paths using
|
||||
`references/claude-code-sessions.md`, `references/codex-sessions.md`,
|
||||
or `references/other-harnesses.md` for any other harness. Confirm a
|
||||
past session by quoting its first prompt and timestamp, and list every
|
||||
candidate you rejected with the reason, or "none". Enumerate subagent
|
||||
transcripts. Create
|
||||
`~/.superpowers/diagnosing-superpowers/<session-id>/`, tell your
|
||||
partner the path, and fill `templates/case.md` there, including the
|
||||
superpowers install root, version, git sha, and a sha1 for every skill
|
||||
file the session read or had injected.
|
||||
3. **Triage.** Read the region around the reported problem yourself. Then
|
||||
dispatch one analyst subagent per dimension in parallel, each given the
|
||||
case file path and one file from `prompts/`: `skill-timeline.md`,
|
||||
`plan-adherence.md`, `repeated-work.md`, `stumbles.md`,
|
||||
`quality-evidence.md`, `request-conflicts.md`, `cost-and-time.md`.
|
||||
Split a dimension by turn range when the transcript is long. Discard
|
||||
any returned finding without `path:line`.
|
||||
4. **Report.** Fill every section of `templates/report.md` in order, write
|
||||
it to the workspace, show it, and give the path.
|
||||
5. **GitHub issues** — when report §7 says possible or likely, or your
|
||||
partner asks. Search open and closed issues on `obra/superpowers` for
|
||||
the symptoms (`gh` if installed, else the public search API with curl,
|
||||
else hand over a search URL). Show matches and suggest adding the
|
||||
report to the closest. If none match, draft `templates/issue.md`, show
|
||||
the exact text, and create it only after approval. `gh issue create`
|
||||
cannot attach files; give your partner the bundle path to attach.
|
||||
6. **Export** — when asked, or the intake goal was a bug report. Ask the
|
||||
redaction level: skeleton, evidence, or full. Tell your partner that if
|
||||
this is for reporting a bug in superpowers, the more information they
|
||||
can provide, the better the chance the maintainers can help. Build the
|
||||
bundle per `templates/bundle-README.md`, dispatch `prompts/scrub.md`, then
|
||||
`prompts/scrub-audit.md`, repeating both until the audit returns CLEAN.
|
||||
Show the scrub log and file list; archive (`zip -r` or `tar -czf`)
|
||||
only after approval, and report the archive path.
|
||||
7. **Similar sessions** — when asked. Turn confirmed findings into a
|
||||
signature, list candidates by mtime and size, find marker line numbers,
|
||||
dispatch `prompts/similar-session.md` per candidate in parallel, and
|
||||
append report §9.
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Complaint | Start with |
|
||||
|---|---|
|
||||
| "It took too long" | cost-and-time, stumbles |
|
||||
| "Why did it do this extra work?" | repeated-work, plan-adherence |
|
||||
| "Why is it so expensive?" | cost-and-time |
|
||||
| "What the hell is it doing?" (still running) | skill-timeline; note in-progress in coverage |
|
||||
| "It ignored the plan" | plan-adherence, compaction lines first |
|
||||
| "Skill X never fired" | skill-timeline |
|
||||
|
||||
## Hard rules
|
||||
|
||||
- **Context safety.** One transcript line can be a megabyte. Check
|
||||
`wc -lc` and long lines first. Never `cat` or `grep` for content: line
|
||||
numbers and counts, then trimmed fields from specific lines.
|
||||
- **Read-only.** Never modify, move, or delete a session file.
|
||||
- **Exact paths to subagents.** A subagent's "current session" is its
|
||||
own. Pass absolute paths and ids.
|
||||
- **Human prompts only.** Hook output, system reminders, and tool results
|
||||
are not your partner's words. In a subagent transcript, "user" is the
|
||||
parent agent.
|
||||
- **No superpowers diagnosis.** Report §7 states involvement and stops.
|
||||
Never name a defect in a skill or propose a change. Pushing does not
|
||||
waive this; point at the issue step and offer the bundle. No advice to
|
||||
your partner either.
|
||||
- **Approval gates.** No archive before your partner has seen the scrub
|
||||
log and file list. No issue or comment before they approve the exact
|
||||
text.
|
||||
- **Intake before analysis.** Nothing in steps 2–7 starts until your
|
||||
partner has answered. If they are away, write the questions and stop.
|
||||
A statement you reconstructed for them is not an answer. An
|
||||
already-scoped request — one specific event, what is running now, or
|
||||
the analysis to run — is itself the statement: answer it, then ask.
|
||||
A whole-session "why" is a complaint.
|
||||
|
||||
## Red Flags
|
||||
|
||||
| Thought | Reality |
|
||||
|---------|---------|
|
||||
| "The problem is obvious, skip intake" | The problem statement scopes everything. Ask. |
|
||||
| "They're away, so I'll reconstruct the statement" | You cannot reconstruct what they wanted. Write the questions and stop. |
|
||||
| "I'll sweep everything now and ask at the end" | An unscoped sweep spends their budget on the wrong question. Ask first. |
|
||||
| "Small, targeted edit, no restructuring needed" | Not your call, however small. Report the evidence; the triager decides. |
|
||||
| "The price per token is well known" | Numbers you did not compute from the transcript are invented. Cite or drop. |
|
||||
@@ -1,65 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Cost and time
|
||||
|
||||
Account for where tokens and wall-clock went.
|
||||
|
||||
1. Tokens. Claude Code: sum `message.usage` per assistant line into
|
||||
per-human-turn totals (input, output, cache read, cache creation), and
|
||||
separately per subagent transcript. Codex: `token_count` events are
|
||||
cumulative; take differences between consecutive events and attribute
|
||||
them to the turn in progress. Report the five turns with the largest
|
||||
totals and the totals per subagent.
|
||||
2. Wall-clock. Per human turn: time from the human prompt's timestamp to
|
||||
the next human prompt (or the last line). Codex also has
|
||||
`task_complete.duration_ms`. Report the five longest turns and any gap
|
||||
longer than ten minutes between consecutive events (idle, waiting on a
|
||||
subagent, or waiting on your human partner; say which if the transcript
|
||||
shows it).
|
||||
3. Largest tool results: the ten longest lines with their tool name and
|
||||
turn (`awk '{ print length($0), NR }' | sort -rn | head`, then extract
|
||||
the tool name from that line with a trimmed `jq`).
|
||||
4. Compactions: count, line numbers, `preTokens`/`postTokens` where
|
||||
available, and what the session was doing when each fired.
|
||||
5. Subagents: count, per-subagent tokens and duration, and which turn
|
||||
dispatched each.
|
||||
6. Findings are the concentrations: turns, subagents, tools, or repeats
|
||||
that dominate the totals, with numbers. Do not speculate about why a
|
||||
turn was expensive beyond what the transcript shows.
|
||||
@@ -1,66 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Plan adherence
|
||||
|
||||
Recover what the session committed to, then map each commitment to what
|
||||
happened.
|
||||
|
||||
1. Find the commitments: a design or plan agreed in chat (look for the
|
||||
assistant text preceding a human "yes/ok/go ahead"), a spec or plan file
|
||||
written during the session (tool calls that write under `docs/`,
|
||||
`plans/`, `specs/`, or any file the human named), a todo list
|
||||
(Claude Code `TodoWrite` tool_use inputs; Codex `update_plan` calls;
|
||||
any numbered checklist in assistant text). Quote each commitment with
|
||||
its `path:line`.
|
||||
2. Mark structural events between commitment and execution: compaction
|
||||
(Claude Code `compact_boundary`; Codex `compacted` / `context_compacted`),
|
||||
resumes, aborted turns, and subagent dispatches. Note their line
|
||||
numbers; plan drift right after one of these is a distinct finding.
|
||||
3. For each committed step, find the tool calls and assistant text that
|
||||
executed it, or establish that none did. Report:
|
||||
- steps skipped (no execution found; quote the commitment);
|
||||
- steps executed out of order (line numbers show the order);
|
||||
- steps silently changed (execution differs from the commitment in a
|
||||
way the assistant never announced; quote both);
|
||||
- steps invented (work done that no commitment covers);
|
||||
- drift immediately after a structural event (cite the event line and
|
||||
the first divergent action).
|
||||
4. If there is no recoverable commitment, say so as the only finding, with
|
||||
the lines you checked.
|
||||
@@ -1,62 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Quality evidence
|
||||
|
||||
Judge the process against its own claims. This is not a code review; do
|
||||
not evaluate the code the session produced.
|
||||
|
||||
1. Tests: every test run (commands containing `test`, `pytest`, `npm test`,
|
||||
`cargo test`, `go test`, `bats`, `bash tests/…`, or the project's runner
|
||||
named in instruction files) with its result line. Report runs that
|
||||
failed and what the assistant did next.
|
||||
2. Verification behind claims: find assistant text claiming done, fixed,
|
||||
passing, verified, works, complete. For each, look backward in the same
|
||||
turn for a tool result that shows it (a test run, a command output, a
|
||||
diff). Report claims with no supporting result in that turn.
|
||||
3. Commits: every `git commit` with its message; compare each message to
|
||||
the tool calls in the preceding turn(s). Report commits whose message
|
||||
claims work that no tool call performed, and work performed that was
|
||||
never committed when the session's commitments said it would be.
|
||||
4. Review feedback: where a reviewer (human or subagent) raised points,
|
||||
find the response. Report points acknowledged but not acted on, and
|
||||
points dismissed without a stated reason.
|
||||
5. Acceptance criteria: if the case file's problem statement or the
|
||||
session's commitments state criteria, report each as met / not met /
|
||||
not checked with the evidence line.
|
||||
@@ -1,63 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Repeated work
|
||||
|
||||
Find work the session did more than once.
|
||||
|
||||
1. Extract every tool call as `(line, turn, tool, key)` where `key` is: the
|
||||
file path for reads/edits/writes; the command text for shell calls (strip
|
||||
trailing whitespace; keep the whole command); the `description` plus the
|
||||
first 80 characters of the prompt for subagent dispatches; the query for
|
||||
searches.
|
||||
2. Group by `(tool, key)`. Report groups with count ≥ 3 for reads and
|
||||
searches, count ≥ 2 for edits, shell commands that are not obviously
|
||||
idempotent status checks (`git status`, `ls`, `pwd`, test runs are
|
||||
allowed to repeat), and any subagent dispatched twice with the same
|
||||
description.
|
||||
3. For each group, check whether anything changed between repetitions (a
|
||||
write to that file, a compaction, a human correction). Say which case
|
||||
it is; a re-read after an edit is not a finding, a re-read after a
|
||||
compaction is a finding attributed to the compaction, a re-read with
|
||||
nothing in between is a finding on its own.
|
||||
4. Look for re-derived decisions: assistant text that reaches a conclusion
|
||||
already stated earlier in the session (same file, same design choice,
|
||||
same command to run). Quote both places.
|
||||
5. One finding per group, with the first and last line numbers and the
|
||||
count.
|
||||
@@ -1,60 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Request conflicts
|
||||
|
||||
Only human-typed prompts count. Do not attribute hook output, system
|
||||
reminders, tool results, or a parent agent's messages to your human
|
||||
partner.
|
||||
|
||||
1. List every human prompt with line and turn. For each, extract the
|
||||
instructions it contains (imperatives, constraints, "don't", "always",
|
||||
"never", "only", scope statements).
|
||||
2. Report:
|
||||
- two human instructions that cannot both be followed (quote both, with
|
||||
lines), and what the assistant did;
|
||||
- a human instruction that conflicts with an instruction file loaded in
|
||||
the session (CLAUDE.md, AGENTS.md, GEMINI.md, or the harness's
|
||||
equivalent; paths are in the case file), quoting both;
|
||||
- a human instruction to skip, ignore, or override a step, skill, or
|
||||
rule, and what happened afterwards;
|
||||
- an instruction the assistant asked to clarify and the answer, when the
|
||||
answer changed scope.
|
||||
3. Do not judge whether your human partner was right. Report the conflict
|
||||
and the assistant's resolution.
|
||||
@@ -1,38 +0,0 @@
|
||||
You are the scrub auditor. Another agent has already scrubbed every file
|
||||
under BUNDLE. Your only job is to find what it missed. You do not fix
|
||||
anything; you report.
|
||||
|
||||
Inputs:
|
||||
- BUNDLE: absolute path of the bundle directory.
|
||||
- PUBLIC_REPOS and PROPRIETARY: same lists the scrubber had.
|
||||
|
||||
Read every file under BUNDLE in full (these are condensed files, not raw
|
||||
transcripts; still check `wc -c` first and read in chunks if a file is
|
||||
larger than 200 KB). Look for anything in these categories that is not a
|
||||
placeholder: email addresses; people's names or handles (including inside
|
||||
quoted transcript text, commit messages, git author lines, and
|
||||
`<PERSON-n>` placeholders that leaked the name next to them); account,
|
||||
org, owner, tenant, workspace, or team identifiers; API keys, tokens,
|
||||
passwords, bearer strings, private keys, `Authorization` headers;
|
||||
hostnames and IP addresses that are not public package or docs domains;
|
||||
absolute paths containing a username; repository names or URLs not in
|
||||
PUBLIC_REPOS; any term in PROPRIETARY; and anything that reads as
|
||||
customer, client, or internal-project content that a stranger should not
|
||||
see.
|
||||
|
||||
Return exactly one of:
|
||||
|
||||
```
|
||||
CLEAN
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```
|
||||
MISSED
|
||||
- <file>:<line> — <category> — <first 20 characters of the value>
|
||||
...
|
||||
```
|
||||
|
||||
Do not paste more than 20 characters of any missed value. Do not comment
|
||||
on the scrub's quality. Do not suggest fixes.
|
||||
@@ -1,38 +0,0 @@
|
||||
You are the scrubber. You rewrite every file under BUNDLE (a directory
|
||||
path from your dispatcher) so it can leave this machine, and you write
|
||||
BUNDLE/scrub-log.md. You never touch anything outside BUNDLE.
|
||||
|
||||
Inputs:
|
||||
- BUNDLE: absolute path of the bundle directory.
|
||||
- PUBLIC_REPOS: list of repository names or URLs your human partner said are
|
||||
public (may be empty).
|
||||
- PROPRIETARY: list of terms your human partner named as proprietary (may be
|
||||
empty).
|
||||
|
||||
Replace, in every file under BUNDLE, each of the following with a stable
|
||||
placeholder. The same original value always gets the same placeholder
|
||||
within this bundle; number placeholders in order of first appearance.
|
||||
|
||||
| Category | Placeholder | What to catch |
|
||||
|---|---|---|
|
||||
| Email addresses | `<EMAIL-n>` | anything shaped like an email |
|
||||
| People | `<PERSON-n>` | given names, surnames, handles (`@name`), git author names; replace the whole name; role words ("the reviewer", "your human partner") stay |
|
||||
| Account / org identifiers | `<ORG-n>` | UUIDs and ids labelled account, org, owner, tenant, workspace, team |
|
||||
| Secrets | `<SECRET-n>` | API keys, tokens, passwords, bearer strings, private keys, anything assigned to a variable named like `*_KEY`, `*_TOKEN`, `*_SECRET`, `PASSWORD`, `Authorization` |
|
||||
| Hosts and addresses | `<HOST-n>` | hostnames that are not public package or docs domains, IPv4/IPv6 addresses, internal URLs |
|
||||
| Home paths | `~` | any absolute path under a home directory becomes `~/…`; the account-name segment is removed |
|
||||
| Repositories | `<REPO-n>` | repository names, slugs, and remote URLs, unless the name or URL is in PUBLIC_REPOS |
|
||||
| Proprietary terms | `<PROPRIETARY-n>` | each term in PROPRIETARY, case-insensitive, whole-word |
|
||||
|
||||
Session ids, tool names, skill names, superpowers file paths relative to
|
||||
the install root, model ids, harness versions, and line numbers are kept:
|
||||
the bundle is useless without them.
|
||||
|
||||
Procedure:
|
||||
1. `find BUNDLE -type f` and process every file, including
|
||||
`environment.json` and `findings/*.md`.
|
||||
2. Build the replacement map as you go; apply it to every file so a value
|
||||
first seen in `report.md` is also replaced in `transcripts/`.
|
||||
3. Write BUNDLE/scrub-log.md: a table of placeholder → category → number of
|
||||
occurrences. Never write the original value into the log.
|
||||
4. Return the scrub-log table and the list of files rewritten. Nothing else.
|
||||
@@ -1,37 +0,0 @@
|
||||
You are a matcher. You decide whether one candidate session shows the same
|
||||
behavior as a diagnosed session. You do not modify any file.
|
||||
|
||||
Inputs:
|
||||
- CASE: absolute path of the diagnosed session's case file. Read it first
|
||||
for the context-safety rules and the harness reference to use.
|
||||
- CANDIDATE: absolute path of one session transcript to examine.
|
||||
- SIGNATURE: a list of markers. Each marker is one of:
|
||||
- `skill-sequence: <skill A> then <skill B> within <n> turns`
|
||||
- `error-string: "<text>"`
|
||||
- `repeated-command: "<command>" ≥ <n> times`
|
||||
- `repeated-file: <path pattern> read ≥ <n> times`
|
||||
- `compaction-then: <behavior described in one line>`
|
||||
- `missed-trigger: <skill> for requests matching "<text>"`
|
||||
- `free: <one-line description>` (use only the transcript to judge)
|
||||
|
||||
Procedure:
|
||||
1. `wc -lc` and the long-line check on CANDIDATE. Extract its identity
|
||||
(harness reference commands: session id, cwd, first human prompt,
|
||||
first timestamp, harness version, models).
|
||||
2. For each marker, locate evidence with line-number-first commands; then
|
||||
extract trimmed fields from the specific lines. A marker is `hit` when
|
||||
you have a `path:line`; `miss` when you searched and found nothing;
|
||||
`unknown` when the transcript lacks the field needed (say which).
|
||||
3. Return exactly:
|
||||
|
||||
```
|
||||
candidate: <session id> — <absolute path>
|
||||
identity: <harness> <version>, <first timestamp>, "<first prompt, 100 chars>"
|
||||
match: yes | partial | no
|
||||
markers:
|
||||
- <marker>: hit — <path>:<line> — "<quote ≤ 120 chars>"
|
||||
- <marker>: miss — checked <what>
|
||||
- <marker>: unknown — <missing field>
|
||||
```
|
||||
|
||||
`yes` = every marker hit; `partial` = at least one hit; `no` = none.
|
||||
@@ -1,69 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Skill timeline
|
||||
|
||||
Build the per-human-turn record of skill and plugin use, then look for gaps.
|
||||
|
||||
1. List the human prompts with line numbers and timestamps.
|
||||
2. List every skill invocation (Claude Code: `Skill` tool_use `input.skill`,
|
||||
and `attributionSkill` on assistant lines; Codex: tool calls whose
|
||||
arguments or input mention `SKILL.md`; other harnesses: reads of files
|
||||
named `SKILL.md`). Record the line, the skill name, and the human turn
|
||||
it happened in.
|
||||
3. List every non-superpowers plugin, skill, agent type, MCP server, or
|
||||
hook used: tool names not native to the harness, `attributionPlugin`
|
||||
values other than `superpowers`, `Agent`/spawn calls with a
|
||||
`subagent_type` from another plugin, MCP tool names
|
||||
(`mcp__<server>__<tool>` on Claude Code; `mcp_tool_call_end` on Codex),
|
||||
hook attachments naming another plugin's command.
|
||||
4. For each human turn, compare the request text against the trigger
|
||||
descriptions of the superpowers skills installed (read
|
||||
`<install root>/skills/*/SKILL.md` frontmatter `description` lines; the
|
||||
install root is in the case file). Report as findings:
|
||||
- a skill invoked, with the request that preceded it (one finding per
|
||||
invocation is fine when there are few; group by skill when many);
|
||||
- a turn whose request matches a skill's trigger description with no
|
||||
invocation in that turn (state which description matched and quote
|
||||
the request);
|
||||
- a skill invoked one or more turns after the matching request (late);
|
||||
- each non-superpowers plugin/skill/tool used, with where.
|
||||
|
||||
Do not say whether a missed or late trigger was wrong. Report the match
|
||||
and the absence; the reader decides.
|
||||
@@ -1,66 +0,0 @@
|
||||
You are an analyst subagent. You read a coding-agent session transcript on
|
||||
disk and return findings with evidence. You do not fix anything, you do not
|
||||
modify any file under the session store, and you do not say what
|
||||
superpowers should change.
|
||||
|
||||
Inputs (from your dispatcher):
|
||||
- CASE: absolute path of the case file. Read it first. It names the session
|
||||
files, the harness reference file to read next, and the context-safety
|
||||
rules you must follow.
|
||||
- RANGE (optional): a turn range or line range. If present, analyze only
|
||||
that range and say so in your Checked line.
|
||||
|
||||
Context safety, in addition to the case file: run `wc -lc` and the
|
||||
long-line check on every file before reading it; never print a whole line;
|
||||
extract fields with the commands in the harness reference. If a command
|
||||
returns more than 500 characters for one record, narrow it. "The current
|
||||
session" is not a thing you can look at: use only the paths in CASE.
|
||||
|
||||
Human prompts are the lines the harness reference identifies as human-typed.
|
||||
Hook output, system reminders, and tool results are not human prompts. In a
|
||||
subagent transcript, "user" is the parent agent.
|
||||
|
||||
Return format (nothing else):
|
||||
|
||||
```
|
||||
## <Dimension> findings
|
||||
|
||||
- finding: <one sentence, what happened>
|
||||
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
|
||||
turns: <first human turn>–<last human turn>
|
||||
confidence: high | medium | low
|
||||
|
||||
Checked: <what you examined: files, line ranges, commands used>
|
||||
```
|
||||
|
||||
A finding without a `path:line` will be discarded by the dispatcher, so do
|
||||
not write one. If you found nothing, return `- none found` and the Checked
|
||||
line.
|
||||
|
||||
Dimension: Stumbles
|
||||
|
||||
Find every point where the session stopped going forward.
|
||||
|
||||
Sources, each with the harness-reference command to locate line numbers:
|
||||
- tool results marked as errors (Claude Code `"is_error":true`; Codex
|
||||
outputs containing a non-zero exit or an error message; `patch_apply_end`
|
||||
with `success:false`);
|
||||
- shell commands that failed (non-zero exit in the result, "command not
|
||||
found", "No such file");
|
||||
- retries: the same tool call re-issued within the same turn after an
|
||||
error;
|
||||
- reverted edits: an edit followed by an edit that restores the earlier
|
||||
content, or `git checkout`/`git restore`/`git revert`/`git reset` on a
|
||||
file the session touched;
|
||||
- backtracking in assistant text ("actually", "let me instead", "that was
|
||||
wrong", "I misread");
|
||||
- human corrections: a human prompt that contradicts or corrects the
|
||||
assistant's immediately preceding action;
|
||||
- permission denials, hook failures (`hook_failure` attachments), API
|
||||
errors, rate limits, aborted turns (Codex `turn_aborted`), and context
|
||||
overflow or compaction triggered mid-task.
|
||||
|
||||
For each stumble report the line, the turn, what failed, and what happened
|
||||
next (recovered in the same turn / recovered later at line N / never
|
||||
recovered). Group identical repeated failures into one finding with a
|
||||
count.
|
||||
@@ -1,105 +0,0 @@
|
||||
# Claude Code session store
|
||||
|
||||
Verified against: Claude Code 2.1.247 (transcript `version` field), macOS.
|
||||
When a field below is missing from the file in front of you, trust the file
|
||||
and say so in coverage notes.
|
||||
|
||||
## Where
|
||||
|
||||
- Main transcript: `~/.claude/projects/<cwd-slug>/<sessionId>.jsonl`, where
|
||||
`<cwd-slug>` is the working directory with every `/` replaced by `-`
|
||||
(e.g. `/tmp/work` → `-tmp-work`).
|
||||
- Subagent transcripts: `~/.claude/projects/<cwd-slug>/<sessionId>/subagents/agent-<agentId>.jsonl`,
|
||||
each with a sibling `agent-<agentId>.meta.json`
|
||||
(`agentType`, `description`, `toolUseId`, `spawnDepth`, optional `model`).
|
||||
- Plugin registry: `~/.claude/plugins/installed_plugins.json` — per plugin:
|
||||
`installPath`, `version`, `installedAt`, `lastUpdated`, `gitCommitSha`.
|
||||
- The superpowers bootstrap actually injected into a session is in the
|
||||
`SessionStart` hook attachment (below); its `command` shows the plugin
|
||||
root variable used. A dev checkout loaded with `--plugin-dir` will not be
|
||||
in the registry, so report both the registry entry and the hook evidence.
|
||||
|
||||
## Which file is the current session
|
||||
|
||||
The most recently modified `.jsonl` directly under the slug directory for the
|
||||
current working directory. Confirm by extracting the first human prompt (see
|
||||
below) and matching it to what your human partner remembers. If two files
|
||||
are close in mtime, show both first prompts and ask.
|
||||
|
||||
## Line types
|
||||
|
||||
Every line is one JSON object. `type` values seen: `user`, `assistant`,
|
||||
`attachment`, `system`, plus session-level records (`permission-mode`,
|
||||
`mode`, `bridge-session`, `last-prompt`, `ai-title`, `atis-latch`,
|
||||
`pr-link`, `queue-operation`, `relocated`, `worktree-state`).
|
||||
|
||||
Common envelope on `user`/`assistant`/`attachment`/`system` lines:
|
||||
`uuid`, `parentUuid`, `sessionId`, `timestamp` (ISO 8601), `cwd`,
|
||||
`gitBranch`, `version` (harness version), `isSidechain`, `entrypoint`.
|
||||
|
||||
| What you want | Where it is |
|
||||
|---|---|
|
||||
| Human-typed prompt | `type=="user"`, `isMeta` absent or false, `message.content` is a string or a list whose first block is `type:"text"`. Lines whose first block is `tool_result` are tool results, not prompts. `<system-reminder>` text inside a prompt is injected, not typed. Text beginning with `<task-notification>`, `<command-name>`, `<local-command-stdout>`, `<system-reminder>`, or `This session is being continued from a previous conversation` is harness-injected too, even though `isMeta` is absent on those lines — exclude them or your human-turn count will be several times too high. |
|
||||
| Human-typed prompt queued mid-turn | `type=="attachment"`, `attachment.type=="queued_command"`, `attachment.origin.kind=="human"`, text in `attachment.prompt`. These are typed while a turn is running and never appear as standalone `user` lines, so they are missing from the list above. Add them to the timeline. |
|
||||
| Assistant text / tool calls | `type=="assistant"`, `message.content[]` blocks of `type:"text"` or `type:"tool_use"` (`id`, `name`, `input`). |
|
||||
| Tool result | `type=="user"`, `message.content[0].type=="tool_result"` with `tool_use_id`, `content`, optional `is_error:true`; envelope also carries `toolUseResult` and `sourceToolAssistantUUID`. |
|
||||
| Model | `message.model` on assistant lines. |
|
||||
| Tokens | `message.usage` on assistant lines: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`. |
|
||||
| Skill invocation | `tool_use` block with `name:"Skill"` and `input.skill` (e.g. `superpowers:brainstorming`); the tool result line has `toolUseResult.commandName`. |
|
||||
| Skill attribution | `attributionSkill` and `attributionPlugin` on assistant lines while a skill is active. |
|
||||
| Subagent dispatch | `tool_use` with `name:"Agent"` (`input.description`, `input.subagent_type`, `input.prompt`); the subagent's own file is matched by `toolUseId` in its `.meta.json`. Subagent lines have `isSidechain:true` and `agentId`. |
|
||||
| Hook output | `type=="attachment"`, `attachment.type` `hook_success`/`hook_failure`, `attachment.hookName` (e.g. `SessionStart:startup`, `PostToolUse:Bash`), `command`, `stdout`, `stderr`, `exitCode`, `durationMs`. |
|
||||
| Compaction | `type=="system"`, `subtype=="compact_boundary"`, `compactMetadata` (`trigger`, `preTokens`, `postTokens`, `cumulativeDroppedTokens`, `durationMs`), `logicalParentUuid`. |
|
||||
| Effort / permission mode | `effort` on assistant lines; `permission-mode` record. |
|
||||
|
||||
## Safe extraction
|
||||
|
||||
Lines can exceed a megabyte. Never print a whole line. Check size first:
|
||||
|
||||
```bash
|
||||
F=~/.claude/projects/<slug>/<id>.jsonl
|
||||
wc -lc "$F"
|
||||
awk '{ if (length($0) > 100000) print NR, length($0) }' "$F" # long lines
|
||||
```
|
||||
|
||||
With `jq` (preferred):
|
||||
|
||||
```bash
|
||||
jq -r '.type' "$F" | sort | uniq -c # line-type census
|
||||
jq -r 'select(.type=="user" and .isMeta!=true and ((.message.content|type)=="string" or .message.content[0].type=="text"))
|
||||
| select((.message.content|if type=="string" then . else (.[0].text // "") end)
|
||||
| test("^(<task-notification>|<command-name>|<local-command-stdout>|<system-reminder>|This session is being continued)") | not)
|
||||
| "\(input_line_number)\t\(.timestamp)\t\((.message.content|if type=="string" then . else .[0].text end)[0:160])"' "$F" # human prompts
|
||||
jq -r 'select(.type=="attachment" and .attachment.type=="queued_command" and .attachment.origin.kind=="human")
|
||||
| "\(input_line_number)\t\(.timestamp)\t\(.attachment.prompt[0:160])"' "$F" # human prompts queued mid-turn; merge with the list above
|
||||
jq -c 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use")
|
||||
| {name, id, input: (.input|tostring|.[0:120])}' "$F" # tool calls
|
||||
jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use" and .name=="Skill") | .input.skill' "$F" # skill invocations
|
||||
jq -c 'select(.type=="assistant") | {ts:.timestamp, model:.message.model, skill:.attributionSkill,
|
||||
u:(.message.usage|{input_tokens,output_tokens,cache_read_input_tokens,cache_creation_input_tokens})}' "$F" # per-message usage
|
||||
jq -c 'select(.subtype=="compact_boundary") | {line:input_line_number, ts:.timestamp,
|
||||
m:(.compactMetadata|{trigger,preTokens,postTokens,cumulativeDroppedTokens,durationMs})}' "$F" # compactions (full compactMetadata also has UUID lists; keep this trimmed)
|
||||
jq -c 'select(.type=="attachment" and (.attachment.type|startswith("hook"))) | {line:input_line_number, hook:.attachment.hookName, exit:.attachment.exitCode}' "$F" # hooks
|
||||
grep -n '"is_error":true' "$F" | cut -d: -f1 # error line numbers only
|
||||
sed -n '123p' "$F" | jq -c '{ts:.timestamp, first:((.message.content // "") as $c
|
||||
| ($c | if type=="array" then ($c[0] // "") else $c end) | tostring | .[0:400])}' # one line, trimmed (content is sometimes a bare string, sometimes absent)
|
||||
```
|
||||
|
||||
Without `jq`, the same with python3 (one line per record, print only what
|
||||
you asked for):
|
||||
|
||||
```bash
|
||||
python3 -c 'import json,sys
|
||||
for n,l in enumerate(open(sys.argv[1]),1):
|
||||
o=json.loads(l)
|
||||
if o.get("type")=="assistant":
|
||||
for b in o["message"].get("content",[]):
|
||||
if b.get("type")=="tool_use": print(n, b["name"], str(b.get("input"))[:120])' "$F"
|
||||
```
|
||||
|
||||
## Subagents
|
||||
|
||||
List `~/.claude/projects/<slug>/<id>/subagents/`. For each `agent-*.meta.json`
|
||||
print `agentType`, `description`, `model`; the matching `.jsonl` is that
|
||||
subagent's transcript and follows the same line format. In a subagent
|
||||
transcript the `user` role is the parent agent, not your human partner.
|
||||
@@ -1,82 +0,0 @@
|
||||
# Codex session store
|
||||
|
||||
Verified against: Codex CLI 0.146.0, 0.147.0 and 0.149.0-alpha.4.1 rollouts
|
||||
(`cli_version` in `session_meta`), macOS. When a field below is missing from
|
||||
the file in front of you, trust the file and say so in coverage notes.
|
||||
|
||||
## Where
|
||||
|
||||
`~/.codex/sessions/YYYY/MM/DD/rollout-<ISO-timestamp>-<thread-id>.jsonl`.
|
||||
Subagent threads are separate rollout files whose `session_meta.payload`
|
||||
has `thread_source: "subagent"` and `source.subagent.thread_spawn.parent_thread_id`
|
||||
pointing at the parent thread id. Root sessions have `thread_source: "user"`.
|
||||
|
||||
## Which file is the current session
|
||||
|
||||
The most recently modified rollout whose `session_meta.payload.cwd` is the
|
||||
current working directory and whose `thread_source` is `user`. Confirm by
|
||||
matching the first `user_message` event to what your human partner
|
||||
remembers. Newer rollouts may carry no `user_message` event at all: when
|
||||
that command returns nothing, fall back to `response_item` messages with
|
||||
`role:"user"` (see Human-typed prompt below) and confirm against the first
|
||||
of those instead.
|
||||
|
||||
## Line types
|
||||
|
||||
Every line is `{timestamp, type, payload}` (some also carry `ordinal`).
|
||||
`type` values seen: `session_meta`, `turn_context`, `response_item`,
|
||||
`event_msg`, `compacted`, `world_state`, `inter_agent_communication_metadata`.
|
||||
|
||||
| What you want | Where it is |
|
||||
|---|---|
|
||||
| Session identity | `session_meta.payload`: `id`, `session_id`, `cwd`, `originator` (e.g. `Codex Desktop`), `cli_version`, `model_provider`, `thread_source`, `source`, `git` (`commit_hash`, `branch`, `repository_url`), `base_instructions.text`. |
|
||||
| Model per turn | `turn_context.payload`: `turn_id`, `model`, `effort`, `cwd`, `approval_policy`, `sandbox_policy`, `multi_agent_version`. Also `event_msg` `thread_settings_applied`. |
|
||||
| Human-typed prompt | `event_msg` with `payload.type=="user_message"`: `payload.message`. When that returns nothing — seen on `thread_source: "user"` Codex Desktop rollouts at `cli_version 0.149.0-alpha.4.1`, and on subagent rollouts — fall back to `response_item` messages with `payload.role=="user"`, text in `payload.content[0].text`. `role:"developer"` messages are injected boilerplate, not typed, and so is any fallback text that begins with a tag such as `<subagent_notification>`, `<environment_context>`, `<skill>` or `<recommended_plugins>`. On a subagent rollout the fallback text is the parent agent's dispatch prompt, not your human partner's. |
|
||||
| Assistant text | `event_msg` `agent_message` (`payload.message`, `payload.phase`) or `response_item` `message` with `role:"assistant"`. |
|
||||
| Tool calls | `response_item` with `payload.type` `function_call` (`name`, `arguments`, `call_id`) or `custom_tool_call` (`name`, `input`, `call_id`); outputs are `function_call_output` / `custom_tool_call_output` matched by `call_id`. Also `event_msg` `patch_apply_end` (`success`, `changes`), `web_search_end`, `mcp_tool_call_end` (`invocation.server`, `invocation.tool`). |
|
||||
| Turn timing | `event_msg` `task_started` (`turn_id`, `started_at`, `model_context_window`) and `task_complete` (`duration_ms`, `time_to_first_token_ms`, `last_agent_message`); `turn_aborted` (`reason`, `duration_ms`). |
|
||||
| Tokens | `event_msg` `token_count`: `payload.info.total_token_usage` (cumulative; keys include `input_tokens`, `cached_input_tokens`, `output_tokens`) and `payload.rate_limits`. |
|
||||
| Compaction | a `compacted` line (`window_id`, `previous_window_id`, `replacement_history`) and an `event_msg` `context_compacted`. |
|
||||
| Subagents | `event_msg` `sub_agent_activity` (`agent_thread_id`, `agent_path`, `kind`); `response_item` `agent_message` with `author`/`recipient`; the child's own rollout file (see Where). |
|
||||
| Skill use | No attribution field. Look for `SKILL.md` in `function_call.arguments` / `custom_tool_call.input` and in `world_state`/`session_meta` instruction text. |
|
||||
| Reasoning | `response_item` `reasoning` (`summary[].text`; `encrypted_content` is opaque). |
|
||||
|
||||
## Safe extraction
|
||||
|
||||
Rollouts reach hundreds of megabytes; `compacted` lines embed whole
|
||||
histories. Never print a whole line. Check size first:
|
||||
|
||||
```bash
|
||||
F=~/.codex/sessions/YYYY/MM/DD/rollout-....jsonl
|
||||
wc -lc "$F"
|
||||
awk '{ if (length($0) > 100000) print NR, length($0) }' "$F"
|
||||
```
|
||||
|
||||
With `jq`:
|
||||
|
||||
```bash
|
||||
head -1 "$F" | jq '.payload | {id, cwd, originator, cli_version, model_provider, thread_source, git}' # identity
|
||||
jq -r '.type + "/" + (.payload.type // "")' "$F" | sort | uniq -c # census
|
||||
jq -r 'select(.type=="event_msg" and .payload.type=="user_message") | "\(input_line_number)\t\(.timestamp)\t\(.payload.message[0:160])"' "$F" # human prompts
|
||||
jq -r 'select(.type=="response_item" and .payload.type=="message" and .payload.role=="user")
|
||||
| "\(input_line_number)\t\(.timestamp)\t\((.payload.content[0].text // "")[0:160])"' "$F" # human prompts, fallback when the line above returns nothing; skip rows whose text starts with a `<tag>`
|
||||
jq -r 'select(.type=="turn_context") | "\(.timestamp)\t\(.payload.model)\t\(.payload.effort)"' "$F" # model per turn
|
||||
jq -c 'select(.type=="response_item" and (.payload.type=="function_call" or .payload.type=="custom_tool_call"))
|
||||
| {line:input_line_number, name:.payload.name, args:((.payload.arguments // .payload.input)|tostring|.[0:120])}' "$F" # tool calls
|
||||
jq -c 'select(.payload.type=="task_complete" or .payload.type=="turn_aborted") | {ts:.timestamp, type:.payload.type, ms:.payload.duration_ms}' "$F" # turn timing
|
||||
jq -c 'select(.payload.type=="token_count") | {ts:.timestamp, t:.payload.info.total_token_usage}' "$F" # tokens (cumulative)
|
||||
grep -n '"type":"compacted"\|"context_compacted"' "$F" | cut -d: -f1 # compaction line numbers
|
||||
grep -n 'SKILL\.md' "$F" | cut -d: -f1 # skill-read line numbers
|
||||
sed -n '123p' "$F" | jq -c '{ts:.timestamp, type, p:(.payload|tostring|.[0:400])}' # one line, trimmed
|
||||
```
|
||||
|
||||
Find a thread's subagent rollouts (filenames only, never content):
|
||||
|
||||
```bash
|
||||
grep -l '"parent_thread_id":"<thread-id>"' ~/.codex/sessions/*/*/*/rollout-*.jsonl
|
||||
```
|
||||
|
||||
A subagent rollout can carry no `event_msg` `user_message` at all — the
|
||||
parent agent's dispatch prompt instead shows up as a `response_item`
|
||||
`message` with `role:"user"`. If a `user_message` event is present, it is
|
||||
from the parent agent, not your human partner.
|
||||
@@ -1,30 +0,0 @@
|
||||
# Other harnesses: discover, then report what you found
|
||||
|
||||
This file is for any harness without a verified reference in this
|
||||
directory. You know your own harness better than this file does. Use that
|
||||
knowledge, and write down exactly what you found so the report reader can
|
||||
judge it.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Ask the harness.** Many harnesses expose a session or history command
|
||||
(`<harness> session list`, `/sessions`, a "resume" picker). Use it to get
|
||||
the session id and, if shown, the file path.
|
||||
2. **Look under the harness's config directory** (`~/.<harness>/`,
|
||||
`~/.config/<harness>/`, `~/.local/share/<harness>/`) for `sessions`,
|
||||
`history`, `chats`, `threads`, or `projects` directories holding `.jsonl`
|
||||
or `.json` files.
|
||||
3. **Confirm a candidate** by extracting its first human message with a
|
||||
size-safe command (`head -c 2000`, or `jq` on the first record) and
|
||||
matching it to what your human partner remembers. Never print whole
|
||||
lines; treat every candidate like the verified stores: `wc -lc` and a
|
||||
long-line check before anything else.
|
||||
4. **Map the fields you need** by reading a handful of records with `jq -c
|
||||
'keys'` or `head -c`: human prompt, assistant text, tool call and result,
|
||||
model, harness version, timestamps, subagent linkage, compaction.
|
||||
5. **Record in the case file and the report's coverage notes**: the store
|
||||
path, the layout you inferred, which of the fields above you could and
|
||||
could not find, and your confidence. Field-level claims in the report
|
||||
are marked "inferred from the file, not a documented format".
|
||||
6. **If you cannot find the store**, say so and ask your human partner for
|
||||
the path. Do not guess a layout from another harness.
|
||||
@@ -1,44 +0,0 @@
|
||||
# Superpowers session diagnosis bundle
|
||||
|
||||
Session: <session-id>
|
||||
Harness: <name> <version> Superpowers: <version> (<sha or "not a checkout">)
|
||||
Redaction level: skeleton | evidence | full
|
||||
Built: <ISO timestamp>
|
||||
|
||||
## What this is
|
||||
|
||||
A scrubbed record of a coding-agent session in which superpowers was
|
||||
installed and something went wrong, prepared so that an agent or person
|
||||
who was not present can decide whether superpowers contributed and, if so,
|
||||
what to change. The report inside states what happened with `path:line`
|
||||
evidence. By design it contains no diagnosis of superpowers and no proposed
|
||||
fix; that is the reader's job.
|
||||
|
||||
## Files
|
||||
|
||||
- `report.md` — the diagnosis report (problem statement, verdict,
|
||||
environment, sessions, timeline, findings, involvement, coverage notes).
|
||||
- `case.md` — the case file the analysts worked from.
|
||||
- `environment.json` — machine-readable copy of the environment section.
|
||||
- `timeline.md` — the per-turn timeline.
|
||||
- `findings/<dimension>.md` — raw analyst findings per dimension.
|
||||
- `transcripts/<session-id>.md` — condensed per-turn rendering of each
|
||||
examined session (never the raw JSONL). At *skeleton* level tool-result
|
||||
bodies are replaced by `[tool result: <tool>, <bytes> bytes, exit <code>]`;
|
||||
at *evidence* level bodies are kept only for events cited in findings; at
|
||||
*full* level all bodies are kept.
|
||||
- `scrub-log.md` — every placeholder used and its category (never the
|
||||
original value).
|
||||
|
||||
## How to read it
|
||||
|
||||
Start with `report.md` §1–2, then §7 (involvement) and the evidence lines
|
||||
it cites, then the matching turns in `transcripts/`. `path:line` references
|
||||
point at the original files on the reporter's machine; the same line
|
||||
numbers are preserved in the condensed transcripts as `[L<n>]` markers.
|
||||
|
||||
## Redaction
|
||||
|
||||
Placeholders look like `<EMAIL-1>`, `<PERSON-2>`, `<SECRET-3>`, `<HOST-4>`,
|
||||
`<REPO-5>`, `<ORG-6>`, `<PROPRIETARY-7>`; home paths are rewritten to `~/…`. The same placeholder
|
||||
always refers to the same original value within this bundle.
|
||||
@@ -1,50 +0,0 @@
|
||||
# Case: <session-id>
|
||||
|
||||
Workspace: ~/.superpowers/diagnosing-superpowers/<session-id>/
|
||||
Created: <ISO timestamp>
|
||||
|
||||
## Problem statement (agreed with your human partner)
|
||||
|
||||
<One paragraph. Names the session(s), the turn range if known, what was
|
||||
expected, what happened, and the observable that matters: wall-clock,
|
||||
tokens, repeated actions, a specific unexpected action.>
|
||||
|
||||
Goal is a superpowers bug report: yes | no
|
||||
|
||||
## Sessions
|
||||
|
||||
| Role | Session id | Absolute path | Lines | Bytes | Longest line (bytes) | First prompt (first 120 chars) | First timestamp |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| main | | | | | | | |
|
||||
| subagent | | | | | | | |
|
||||
|
||||
Rejected candidates: <id — path — why rejected>, or "none".
|
||||
|
||||
Session still running at read time: yes | no (mtime <ISO>, lines <N>)
|
||||
|
||||
## Environment
|
||||
|
||||
- OS: <name and version>
|
||||
- Harness: <name> <version>
|
||||
- Models seen: <model id — where (main / subagent id)>
|
||||
- Superpowers install root: <path>; version <x.y.z>; git sha <sha or "not a checkout">
|
||||
- Skill files read or injected during the session:
|
||||
|
||||
| File (relative to install root) | sha1 (current file) | mtime newer than session? |
|
||||
|---|---|---|
|
||||
|
||||
- Other plugins / extensions / MCP servers configured: <list, or "none found">
|
||||
- Instruction files present (paths only): <list>
|
||||
|
||||
## Context-safety rules for every reader of these files
|
||||
|
||||
- Check `wc -lc` and long lines (`awk '{ if (length($0) > 100000) print NR, length($0) }'`) before reading.
|
||||
- Never `cat` or `grep` for content. Line numbers and counts first
|
||||
(`grep -n … | cut -d: -f1`), then small fields from specific lines
|
||||
(`sed -n Np | jq -c '{…}'` or `| cut -c1-500`).
|
||||
- Read-only: never modify, move, or delete a session file.
|
||||
- In a subagent transcript, "user" is the parent agent.
|
||||
|
||||
## Harness reference to use
|
||||
|
||||
<references/claude-code-sessions.md | references/codex-sessions.md | references/other-harnesses.md>
|
||||
@@ -1,49 +0,0 @@
|
||||
- [x] I searched existing issues and this is not a duplicate (searched: <query terms>; closest: <#n title, or "none">)
|
||||
|
||||
## Environment (required)
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Superpowers version | <version> (<sha or "not a checkout">) |
|
||||
| Harness (Claude Code, Cursor, etc.) | <harness> |
|
||||
| Harness version | <version> |
|
||||
| Your model + version | <model ids seen> |
|
||||
| All plugins installed | <list> |
|
||||
| OS + shell | <os version>, <shell> |
|
||||
|
||||
## Is this a Superpowers issue or a platform issue?
|
||||
|
||||
- [ ] I confirmed this issue does not occur without Superpowers installed
|
||||
|
||||
Not reproduced without superpowers. Evidence for involvement is below;
|
||||
the reporter has not established cause.
|
||||
|
||||
## What happened?
|
||||
|
||||
<Problem statement, then the triage verdict, with `path:line` citations
|
||||
rewritten as `transcript line <n>`.>
|
||||
|
||||
## Steps to reproduce
|
||||
|
||||
1. <first human prompt, scrubbed>
|
||||
2. <the turns leading to the problem, one line each>
|
||||
3. <the observable>
|
||||
|
||||
## Expected behavior
|
||||
|
||||
<from the problem statement>
|
||||
|
||||
## Actual behavior
|
||||
|
||||
<from the triage verdict>
|
||||
|
||||
## Debug log or conversation transcript
|
||||
|
||||
Session id(s): <ids>. A scrubbed bundle (redaction level: <level>) is
|
||||
attached to this issue by the reporter, or available on request.
|
||||
Superpowers involvement per the diagnosis report: <possible | likely>, with
|
||||
evidence at <transcript lines>. This report does not propose a fix.
|
||||
|
||||
---
|
||||
Filed with the `diagnosing-superpowers` skill. Model, harness, harness
|
||||
version, and installed plugins are listed above.
|
||||
@@ -1,78 +0,0 @@
|
||||
# Session diagnosis: <session-id>
|
||||
|
||||
Report path: ~/.superpowers/diagnosing-superpowers/<session-id>/report.md
|
||||
Written: <ISO timestamp>
|
||||
|
||||
## 1. Problem statement (REQUIRED)
|
||||
|
||||
<Copied from the case file.>
|
||||
|
||||
## 2. Triage verdict (REQUIRED)
|
||||
|
||||
<What the evidence shows happened around the reported problem. Prose, with
|
||||
`path:line` after every claim. State confidence: high / medium / low, and
|
||||
what would raise it. No statement about what superpowers should do.>
|
||||
|
||||
## 3. Environment (REQUIRED)
|
||||
|
||||
- OS:
|
||||
- Harness and version:
|
||||
- Models seen:
|
||||
- Superpowers install root / version / git sha:
|
||||
- Skill files read or injected (sha1 table from the case file):
|
||||
- Other plugins, extensions, MCP servers:
|
||||
- Instruction files present (paths only):
|
||||
|
||||
## 4. Sessions examined (REQUIRED)
|
||||
|
||||
| Role | Session id | Absolute path | Lines | Bytes |
|
||||
|---|---|---|---|---|
|
||||
|
||||
Rejected candidates: <id — path — why>, or "none".
|
||||
|
||||
## 5. Timeline (REQUIRED)
|
||||
|
||||
One row per human-typed prompt. Events column lists skills invoked,
|
||||
subagents dispatched, compaction, errors, resumes, aborts.
|
||||
|
||||
| Turn | Line | Time | Request (one line) | Events |
|
||||
|---|---|---|---|---|
|
||||
|
||||
## 6. Findings (REQUIRED, one subsection per dimension)
|
||||
|
||||
Each finding:
|
||||
```
|
||||
- finding: <one sentence>
|
||||
evidence: <path:line> — "<short quote>"
|
||||
turns: <first>–<last>
|
||||
confidence: high | medium | low
|
||||
```
|
||||
A dimension with nothing to report says `none found — checked: <what was checked>`.
|
||||
|
||||
### 6.1 Skill timeline
|
||||
### 6.2 Plan adherence
|
||||
### 6.3 Repeated work
|
||||
### 6.4 Stumbles
|
||||
### 6.5 Quality evidence
|
||||
### 6.6 Request conflicts
|
||||
### 6.7 Cost and time
|
||||
### 6.8 Other plugins and skills used
|
||||
|
||||
## 7. Superpowers involvement (REQUIRED)
|
||||
|
||||
not indicated | possible | likely
|
||||
|
||||
Evidence lines: <path:line list>. This section states involvement only. It
|
||||
does not name a defect and does not propose a change.
|
||||
|
||||
## 8. Coverage notes (REQUIRED)
|
||||
|
||||
- Not read: <ranges, files, and why>
|
||||
- Harness features unavailable: <list or none>
|
||||
- Session was in progress at read time: yes/no
|
||||
- For your human partner to double-check: <list or none>
|
||||
|
||||
## 9. Similar sessions (only when requested)
|
||||
|
||||
| Session id | Path | Date | Harness | Matched | Did not match |
|
||||
|---|---|---|---|---|---|
|
||||
@@ -173,6 +173,14 @@ its own text agrees with itself — the tests it specifies against the code it
|
||||
specifies, the files it creates against the files it later touches. "The scan
|
||||
is clean" without those rows is not a scan you ran.
|
||||
|
||||
**When the plan's header declares `Plan shape: skeleton-first`,** the
|
||||
table gets a final section: the DISPATCH PLAN — group the pending tasks
|
||||
into waves. Tasks in the same wave are mutually file-disjoint and consume
|
||||
no interface still under construction — dispatch each wave's implementers
|
||||
concurrently, one worktree per task, and integrate before the next wave;
|
||||
tasks that fail those conditions serialize. On a skeleton-first plan, a
|
||||
scan without a dispatch plan is not a scan you ran.
|
||||
|
||||
Write the table to the ledger. Rule on everything you find before execution
|
||||
begins — each finding against the plan text that mandates it — and record
|
||||
each ruling in the ledger. If the scan is clean, proceed without comment.
|
||||
@@ -187,6 +195,10 @@ Use the least powerful model that can handle each role to conserve cost and incr
|
||||
|
||||
**Mechanical implementation tasks** (isolated functions, clear specs, 1-2 files): use a fast, cheap model. Most implementation tasks are mechanical when the plan is well-specified.
|
||||
|
||||
When a task carries a **Tier:** field, follow it — the planner already
|
||||
ruled: mechanical → the cheapest available model; judgment → a standard
|
||||
model. Do not re-litigate the tier at dispatch.
|
||||
|
||||
**Integration and judgment tasks** (multi-file coordination, pattern matching, debugging): use a standard model.
|
||||
|
||||
**Architecture and design tasks**: use the most capable available model.
|
||||
@@ -208,7 +220,10 @@ most expensive — which silently defeats this section.
|
||||
**Turn count beats token price.** Wall-clock and context cost scale with how
|
||||
many turns a subagent takes, and the cheapest models routinely take 2-3× the
|
||||
turns on multi-step work — costing more overall. Use a mid-tier model as the
|
||||
floor for reviewers and for implementers working from prose descriptions.
|
||||
floor for reviewers and for implementers working from task contracts or
|
||||
prose descriptions — unless the task's Tier line says mechanical: the
|
||||
planner has already ruled the deliverable fully specified, so treat a
|
||||
mechanical-tier contract like spelled-out content.
|
||||
When the task's plan text contains the complete code to write, the
|
||||
implementation is transcription plus testing: use the cheapest tier for
|
||||
that implementer. Single-file mechanical fixes also take the cheapest tier.
|
||||
@@ -259,7 +274,13 @@ and fix-round diffs need it.
|
||||
know; (4) your resolution of any ambiguity you noticed in the brief;
|
||||
(5) the report-file path and report contract. Exact values (numbers,
|
||||
magic strings, signatures, test cases) appear only in the brief. Never
|
||||
make a subagent read the whole plan file.
|
||||
make a subagent read the whole plan file. When the brief is a contract
|
||||
(goal, success criteria, interfaces) rather than written-out code, item
|
||||
(3) also carries the elaboration the contract leaves to dispatch time:
|
||||
the interfaces as actually built by completed tasks, environment facts
|
||||
and discoveries from earlier reports, and any amendment rulings. There
|
||||
the success criteria name the cases the tests must cover, and the
|
||||
implementer designs its own code and tests within the contract.
|
||||
- **Report file:** name the implementer's report file after the brief
|
||||
(brief `…/task-N-brief.md` → report `…/task-N-report.md`) and put it in
|
||||
the dispatch prompt. The implementer writes the full report there and
|
||||
@@ -280,6 +301,26 @@ and fix-round diffs need it.
|
||||
- Record the implementer's agent identity from the dispatch result —
|
||||
fix-loop rounds 1-3 resume this agent.
|
||||
- Never dispatch multiple implementation subagents in parallel (conflicts).
|
||||
The one exception is a skeleton-first plan whose dispatch plan shows two
|
||||
or more pending tasks mutually file-disjoint with none consuming an
|
||||
interface still under construction. Dispatch those implementers
|
||||
concurrently, each in its own worktree:
|
||||
- Record the integration base commit in the ledger before the first
|
||||
concurrent dispatch.
|
||||
- Create one worktree per concurrent task off that base
|
||||
(`git worktree add <repo-root>/.worktrees/task-<N> -b task-<N>
|
||||
<base>`); each dispatch's `Work from:` is its own worktree, and its
|
||||
BASE is that worktree's HEAD.
|
||||
- Review each task's diff as usual when it reports. Integrate reviewed
|
||||
branches in plan order: merge each into the integration branch
|
||||
(`git merge --no-ff task-<N>`), and run that task's verification
|
||||
commands after each merge.
|
||||
- A merge conflict or post-merge verification failure is that task's
|
||||
fix-loop round 1: rebase the task branch onto the current
|
||||
integration head in its worktree, then resume its implementer
|
||||
there. Never resolve conflicts yourself.
|
||||
- Remove each worktree (`git worktree remove`) once its branch is
|
||||
integrated, and record the integrated range in the ledger as usual.
|
||||
|
||||
Template: [implementer-prompt.md](implementer-prompt.md)
|
||||
|
||||
@@ -438,6 +479,16 @@ message as your other bookkeeping:
|
||||
- `Task <N>: complete (commits <base7>..<head7>, <K> parked)` after a
|
||||
tripped breaker
|
||||
|
||||
**On a skeleton-first plan,** write one plan-check line with the
|
||||
completion line. Re-read the remaining tasks against what this task
|
||||
actually established — interfaces as built, environment facts,
|
||||
discoveries in the report — and append either `Plan holds` or
|
||||
`Amendment: Task <M>: <what changes and why>` to the ledger. An
|
||||
amendment is plan authority applied at the plan layer: from then on the
|
||||
amended text IS the plan's text, and it rides into every affected task's
|
||||
dispatch under item (3). Never dispatch a task whose brief a completed
|
||||
task's report has already invalidated.
|
||||
|
||||
Then mark the todo complete and move on. Never move to the next task while
|
||||
the review has open Critical/Important issues that are neither fixed nor
|
||||
parked-with-ruling at the cap.
|
||||
|
||||
@@ -5,8 +5,11 @@ Use this template when dispatching an implementer subagent.
|
||||
```
|
||||
Subagent (general-purpose):
|
||||
description: "Implement Task N: [task name]"
|
||||
model: [MODEL — REQUIRED: choose per SKILL.md Model Selection; an omitted
|
||||
model silently inherits the session's most expensive one]
|
||||
model: [MODEL — REQUIRED: when the brief carries a Tier line, set from it:
|
||||
mechanical → the cheapest model the subagent tool offers; judgment →
|
||||
a standard mid-tier model. Otherwise choose per SKILL.md Model
|
||||
Selection. An omitted model silently inherits the session's most
|
||||
expensive one]
|
||||
prompt: |
|
||||
You are implementing Task N: [task name]
|
||||
|
||||
|
||||
@@ -22,6 +22,30 @@ Assume they are a skilled developer, but know almost nothing about our toolset o
|
||||
|
||||
If the spec covers multiple independent subsystems, it should have been broken into sub-project specs during brainstorming. If it wasn't, suggest breaking this into separate plans — one per subsystem. Each plan should produce working, testable software on its own.
|
||||
|
||||
## Two Plan Shapes
|
||||
|
||||
Before mapping files, classify the plan's shape and say the
|
||||
classification out loud — "this composes three subsystems, so I'll plan
|
||||
it skeleton-first" — so your human partner can override it:
|
||||
|
||||
- **Task-by-task (default)** — tasks build the feature a component at a
|
||||
time, each step carrying the actual content the engineer needs. Use it
|
||||
for changes to code that already exists, for a spec that touches one
|
||||
subsystem, and whenever the alternative's conditions do not clearly
|
||||
hold. The rest of this skill describes this shape.
|
||||
- **Skeleton-first (alternative)** — Task 1 is the thinnest end-to-end
|
||||
slice through every subsystem the spec composes; later tasks widen it
|
||||
one component at a time, each from a contract rather than written-out
|
||||
code. Use it when the spec composes more than one subsystem AND a
|
||||
running end-to-end slice early is worth a longer total build. Read
|
||||
[skeleton-first-plans.md](skeleton-first-plans.md) before writing one
|
||||
— it adds one line to the plan header and replaces this skill's task
|
||||
granularity, task template, and plan-failure list.
|
||||
|
||||
When in doubt, plan task-by-task. Skeleton-first buys an earlier running
|
||||
system and pays for it in total wall clock; it is a trade, not an
|
||||
upgrade.
|
||||
|
||||
## File Structure
|
||||
|
||||
Before defining tasks, map out which files will be created or modified and what each one is responsible for. This is where decomposition decisions get locked in.
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
# Skeleton-First Plans
|
||||
|
||||
The alternative plan shape from writing-plans' Two Plan Shapes router.
|
||||
Each section below replaces the same-named section of
|
||||
[SKILL.md](SKILL.md); everything SKILL.md says that is not named here
|
||||
still binds — Scope Check, File Structure, Task Right-Sizing, the plan
|
||||
header, Self-Review, and the Execution Handoff.
|
||||
|
||||
## Overview
|
||||
|
||||
Write a plan that carries the decisions, not the keystrokes:
|
||||
decomposition, file structure, interfaces, constraints, and a precise
|
||||
contract per task. Assume the engineer is skilled and designs their own
|
||||
code and tests from a precise contract, but knows nothing about our
|
||||
codebase, toolset, or problem domain — every name, path, constraint, and
|
||||
behavior they must match is stated explicitly. DRY. YAGNI. TDD.
|
||||
Frequent commits.
|
||||
|
||||
## When This Shape Fits
|
||||
|
||||
Use it when the spec composes more than one subsystem and a running
|
||||
end-to-end slice early is worth a longer total build: the value arrives
|
||||
as soon as real input reaches real output, and every later task widens
|
||||
something that already runs.
|
||||
|
||||
Do not use it for a change to one subsystem, or when the whole point is
|
||||
to land the finished thing as fast as possible. This shape spends its
|
||||
first task on a slice that does almost nothing, and it spends planning
|
||||
effort on contracts and interfaces the task-by-task shape gets for free
|
||||
by writing the code out.
|
||||
|
||||
## Plan Document Header
|
||||
|
||||
The header is SKILL.md's, plus one line directly under the **Goal:**
|
||||
line, which is how executors know which shape they are running:
|
||||
|
||||
```markdown
|
||||
**Plan shape:** skeleton-first
|
||||
```
|
||||
|
||||
## Walking Skeleton First
|
||||
|
||||
Task 1 builds the thinnest end-to-end slice through every subsystem the
|
||||
spec composes — real input to real output — before any task deepens a
|
||||
single layer; later tasks widen the skeleton.
|
||||
|
||||
The test of a skeleton is that it runs. A first task that builds the
|
||||
data loader, the schema, or the config layer is a foundation, not a
|
||||
skeleton: nothing runs until something above it exists. A skeleton
|
||||
reaches the output — thinly, with one real case — through every
|
||||
subsystem the spec names.
|
||||
|
||||
## Task Contracts, Not Task Scripts
|
||||
|
||||
A task states WHAT must exist when it is done, precisely enough that a
|
||||
skilled engineer can build it without asking you anything, without
|
||||
prescribing HOW:
|
||||
|
||||
- **Goal:** one short paragraph naming the deliverable and its role in
|
||||
the feature.
|
||||
- **Success criteria:** concrete, checkable behaviors — exact commands
|
||||
to run and what they must show, the cases tests must cover (including
|
||||
failure cases), constraints that bind the implementation.
|
||||
- **Notes:** what the engineer needs and cannot discover alone — spec
|
||||
sections to read, files worth reading first, known pitfalls.
|
||||
|
||||
The Interfaces block carries the exact names, signatures, and types;
|
||||
the success criteria carry the behaviors; the engineer supplies the
|
||||
code and the test design. TDD and frequent commits remain required.
|
||||
|
||||
## Task Structure
|
||||
|
||||
````markdown
|
||||
### Task N: [Component Name]
|
||||
|
||||
**Files:**
|
||||
- Create: `exact/path/to/file.py`
|
||||
- Modify: `exact/path/to/existing.py:123-145`
|
||||
- Test: `tests/exact/path/to/test.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: [what this task uses from earlier tasks — exact signatures]
|
||||
- Produces: [what later tasks rely on — exact function names, parameter
|
||||
and return types. A task's implementer sees only their own task; this
|
||||
block is how they learn the names and types neighboring tasks use.]
|
||||
|
||||
**Goal:** [one paragraph — the deliverable and its role in the feature]
|
||||
|
||||
**Success criteria:**
|
||||
- Run: `pytest tests/exact/path/to/test.py -v` — all tests pass; tests
|
||||
cover [the specific behaviors and failure cases, named concretely]
|
||||
- [observable behavior the deliverable must exhibit, with the exact
|
||||
command or input/output that demonstrates it]
|
||||
- [constraint that binds the implementation, copied from the spec]
|
||||
|
||||
**Notes:** [spec sections to read; files to read first; known pitfalls]
|
||||
|
||||
**Tier:** mechanical | judgment. Mechanical = the deliverable is fully
|
||||
specified by Files + Interfaces + success criteria above (most tasks in
|
||||
a well-specified plan are mechanical); judgment = multi-file
|
||||
coordination, debugging, or real design latitude remains. The
|
||||
implementer's model follows this field — mark it deliberately.
|
||||
|
||||
**Commit:** one commit ending the task; message named here.
|
||||
````
|
||||
|
||||
## No Vague Contracts
|
||||
|
||||
Every contract must be checkable by someone who did not write it. These
|
||||
are **plan failures** — never write them:
|
||||
- "TBD", "TODO", "implement later", "fill in details"
|
||||
- Goals naming activity instead of a deliverable ("improve error handling")
|
||||
- Success criteria with no observable check ("works correctly", "handles edge cases")
|
||||
- Interfaces blocks omitting a name, signature, or type another task consumes
|
||||
- "Similar to Task N" (state this task's own contract in full — the engineer may be reading tasks out of order)
|
||||
- References to types, functions, or methods not defined in any task's Interfaces block
|
||||
|
||||
## Self-Review
|
||||
|
||||
Run SKILL.md's Self-Review checklist, reading step 2 against "No Vague
|
||||
Contracts" above rather than "No Placeholders".
|
||||
|
||||
## Red Flags
|
||||
|
||||
| Thought | Reality |
|
||||
|---------|---------|
|
||||
| "Task 1 is the data loader — that's the foundation" | A foundation is a layer. The skeleton runs real input to real output through every subsystem the spec names, thinly. |
|
||||
| "The skeleton can return a hardcoded value for now" | It may be thin, but the path must be real: real input, real wiring, real output. A hardcoded response tests nothing end to end. |
|
||||
| "A contract without the code is vague" | Vague is an uncheckable success criterion. Exact names, exact commands, exact expected output — no code. |
|
||||
| "I'll write the test code into the task to be safe" | The success criteria name the cases; the implementer designs the tests. Written-out tests are the task-by-task shape. |
|
||||
| "Skeleton-first is the better shape, so I'll use it here" | It costs total wall clock. Without more than one subsystem and a reason to want an early running slice, plan task-by-task. |
|
||||
@@ -1,127 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Structural checks for skills/diagnosing-superpowers. Behavior is tested by
|
||||
# scenario evals kept by the maintainer; this script only checks the things a
|
||||
# shell can check: frontmatter, referenced files exist, no local paths or
|
||||
# names leaked into shipped files, SKILL.md word budget.
|
||||
set -u
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
SKILL_DIR="$REPO_ROOT/skills/diagnosing-superpowers"
|
||||
SKILL_MD="$SKILL_DIR/SKILL.md"
|
||||
WORD_BUDGET=900
|
||||
|
||||
PASSES=0
|
||||
FAILURES=0
|
||||
|
||||
pass() { echo " [PASS] $1"; PASSES=$((PASSES + 1)); }
|
||||
fail() { echo " [FAIL] $1"; FAILURES=$((FAILURES + 1)); }
|
||||
|
||||
echo "diagnosing-superpowers structure"
|
||||
|
||||
# --- SKILL.md frontmatter -------------------------------------------------
|
||||
if [ -f "$SKILL_MD" ]; then
|
||||
pass "SKILL.md exists"
|
||||
frontmatter="$(awk 'NR==1 && $0!="---"{exit} NR>1 && $0=="---"{exit} NR>1{print}' "$SKILL_MD")"
|
||||
if printf '%s\n' "$frontmatter" | grep -q '^name: diagnosing-superpowers$'; then
|
||||
pass "frontmatter name is diagnosing-superpowers"
|
||||
else
|
||||
fail "frontmatter name is diagnosing-superpowers"
|
||||
fi
|
||||
description="$(printf '%s\n' "$frontmatter" | awk '/^description:/{sub(/^description:[ ]*/,""); print; found=1; next} found && /^[ ]/{print} found && !/^[ ]/{exit}' | tr '\n' ' ')"
|
||||
if printf '%s' "$description" | grep -q '^Use when'; then
|
||||
pass "description starts with 'Use when'"
|
||||
else
|
||||
fail "description starts with 'Use when' (got: ${description:0:60})"
|
||||
fi
|
||||
if [ "${#description}" -le 1024 ]; then
|
||||
pass "description under 1024 characters"
|
||||
else
|
||||
fail "description under 1024 characters (${#description})"
|
||||
fi
|
||||
for banned in "dispatch" "then" "step"; do
|
||||
if printf '%s' "$description" | grep -qiw "$banned"; then
|
||||
fail "description contains workflow word '$banned'"
|
||||
else
|
||||
pass "description avoids workflow word '$banned'"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- word budget --------------------------------------------------------
|
||||
body_words="$(awk 'BEGIN{fm=0} NR==1 && $0=="---"{fm=1; next} fm==1 && $0=="---"{fm=2; next} fm==2{print}' "$SKILL_MD" | wc -w | tr -d ' ')"
|
||||
if [ "$body_words" -le "$WORD_BUDGET" ]; then
|
||||
pass "SKILL.md body within $WORD_BUDGET words ($body_words)"
|
||||
else
|
||||
fail "SKILL.md body within $WORD_BUDGET words ($body_words)"
|
||||
fi
|
||||
|
||||
# --- required sections --------------------------------------------------
|
||||
for heading in "## Hard rules" "## Red Flags"; do
|
||||
if grep -q "^$heading" "$SKILL_MD"; then
|
||||
pass "SKILL.md has section '$heading'"
|
||||
else
|
||||
fail "SKILL.md has section '$heading'"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- every referenced skill file exists --------------------------------
|
||||
while IFS= read -r ref; do
|
||||
if [ -f "$SKILL_DIR/$ref" ]; then
|
||||
pass "referenced file exists: $ref"
|
||||
else
|
||||
fail "referenced file exists: $ref"
|
||||
fi
|
||||
done < <(grep -o '\(references\|prompts\|templates\)/[A-Za-z0-9._-]*\.md' "$SKILL_MD" | sort -u)
|
||||
else
|
||||
fail "SKILL.md exists"
|
||||
fi
|
||||
|
||||
# --- expected files -------------------------------------------------------
|
||||
expected_files=(
|
||||
references/claude-code-sessions.md
|
||||
references/codex-sessions.md
|
||||
references/other-harnesses.md
|
||||
prompts/skill-timeline.md
|
||||
prompts/plan-adherence.md
|
||||
prompts/repeated-work.md
|
||||
prompts/stumbles.md
|
||||
prompts/quality-evidence.md
|
||||
prompts/request-conflicts.md
|
||||
prompts/cost-and-time.md
|
||||
prompts/scrub.md
|
||||
prompts/scrub-audit.md
|
||||
prompts/similar-session.md
|
||||
templates/case.md
|
||||
templates/report.md
|
||||
templates/bundle-README.md
|
||||
templates/issue.md
|
||||
)
|
||||
for rel in "${expected_files[@]}"; do
|
||||
if [ -f "$SKILL_DIR/$rel" ]; then
|
||||
pass "expected file present: $rel"
|
||||
else
|
||||
fail "expected file present: $rel"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- no local paths or names in shipped files ----------------------------
|
||||
leaks="$(grep -rn -E '/Users/|/home/|jesse' "$SKILL_DIR" "$SCRIPT_DIR" --exclude=test-skill-structure.sh 2>/dev/null || true)"
|
||||
if [ -z "$leaks" ]; then
|
||||
pass "no machine-specific paths or names in shipped files (skills + tests)"
|
||||
else
|
||||
fail "no machine-specific paths or names in shipped files (skills + tests)"
|
||||
printf '%s\n' "$leaks" | head -10 | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
# --- "the user" never appears in skill prose -----------------------------
|
||||
user_hits="$(grep -rn -i 'the user' "$SKILL_DIR" --include='*.md' 2>/dev/null || true)"
|
||||
if [ -z "$user_hits" ]; then
|
||||
pass "skill files say 'your human partner', not 'the user'"
|
||||
else
|
||||
fail "skill files say 'your human partner', not 'the user'"
|
||||
printf '%s\n' "$user_hits" | head -10 | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Passed: $PASSES Failed: $FAILURES"
|
||||
[ "$FAILURES" -eq 0 ]
|
||||
Reference in New Issue
Block a user