Compare commits

..

3 Commits

Author SHA1 Message Date
Jesse Vincent a56a34365a feat(sdd): follow a task's declared implementation tier at dispatch
The skeleton-first task template carries a Tier field the planner marks
deliberately. Model Selection now follows it instead of re-deciding at
dispatch, the implementer template's model slot reads it at fill time, and
an explicit mechanical ruling overrides the mid-tier contract floor —
the planner has already ruled the deliverable fully specified.

Task templates without a Tier field are unaffected: every route falls
through to the existing Model Selection guidance.
2026-08-19 17:32:55 +00:00
Jesse Vincent ef0ca09658 feat(sdd): execute skeleton-first plans with a wave dispatch plan
A plan whose header declares `Plan shape: skeleton-first` gets three
additions, each placed at the moment the controller already attends to:
the pre-flight scan ends with a dispatch plan grouping file-disjoint
tasks into waves; the never-parallel rule gains one exception with the
worktree integration protocol that makes it safe; each task completion
records a plan-check line so an amendment becomes the plan's text.

Every clause is gated on the declared plan shape. Plans without the
declaration execute exactly as before, including the unqualified
never-dispatch-in-parallel rule.
2026-08-19 17:32:31 +00:00
Jesse Vincent c3a5abcf33 feat(writing-plans): add skeleton-first as a selectable plan shape
Adds a plan-shape router in the idiom of brainstorming's Three Paths:
classify the shape, announce it so the human partner can override, and
default to task-by-task on doubt. The alternative shape's full text —
skeleton ordering, task contracts instead of written-out code, the
contract task template, and its plan-failure list — lives in a sibling
reference file, so the default shape's guidance is unchanged.

Skeleton-first is a trade, not an upgrade: it buys an earlier running
end-to-end slice and pays for it in total wall clock.
2026-08-19 17:31:44 +00:00
28 changed files with 213 additions and 3645 deletions
-1
View File
@@ -294,7 +294,6 @@ Superpowers is built by [Jesse Vincent](https://blog.fsck.com) and the rest of t
**Debugging**
- **systematic-debugging** - 4-phase root cause process (includes root-cause-tracing, defense-in-depth, condition-based-waiting techniques)
- **verification-before-completion** - Ensure it's actually fixed
- **diagnosing-superpowers** - Work out what went wrong in a session, with evidence; export a scrubbed bundle or file an issue
**Collaboration**
- **brainstorming** - Socratic design refinement
File diff suppressed because it is too large Load Diff
@@ -1,516 +0,0 @@
# Diagnosing Superpowers Sessions — Design
Date: 2026-08-27
Status: approved by Jesse (in-session); spec pending review
Branch: `diagnosing-superpowers` off `dev`
## Goal
A core skill, `diagnosing-superpowers`, that a user invokes when a
superpowers session went wrong. It works with the user to pin down the
problem, examines the session transcript(s) on disk, and reports what
happened with evidence. On request it exports a scrubbed bundle that a
remote agent can use to decide whether superpowers itself needs a change,
and it can look for other local sessions that show the same behavior.
The skill reports; it never diagnoses superpowers. Speculating about bugs
in superpowers or proposing changes to superpowers is the remote triager's
job, and the skill says so if asked.
## Scope decisions (settled with Jesse)
- **Pure prose skill for v1.** No shipped scripts. The model does the work,
using subagents aggressively. Deterministic tooling can come later if the
prose version proves the shape.
- **Harness coverage.** Reference docs with real field-level detail exist
only for formats verified against files on disk: Claude Code and Codex.
Every other harness gets a discovery procedure. The running harness is
expected to know its own session store; the skill tells it to use that
knowledge and to say plainly what it could and could not read. No
invented formats.
- **Problem intake first.** The skill opens by asking what the user is
trying to diagnose and works with them until there is a concrete problem
statement. Sweeps run in service of that statement.
- **Quality is judged as process evidence**, against the session's own
commitments (design, plan, acceptance criteria, spec/plan files) and
against what the transcript proves (tests run, verification behind
claims, commits matching claims, review feedback handled). It is not a
code review of the resulting diff.
- **Redaction level is the user's call.** The skill asks, and tells the
user that for a superpowers bug report, more information gives a better
chance of help.
- **Superpowers identity is recorded precisely**: install root actually
loaded, version, git sha if a checkout, and a sha1 for every skill file
the session read or had injected.
- **Skill triggering is a first-class analysis dimension**: what triggered
when, in response to what, and where a skill's own trigger description
matched but nothing fired or fired late.
## Skill layout
```
skills/diagnosing-superpowers/
SKILL.md
references/
claude-code-sessions.md
codex-sessions.md
other-harnesses.md
prompts/
skill-timeline.md
plan-adherence.md
repeated-work.md
stumbles.md
quality-evidence.md
request-conflicts.md
cost-and-time.md
scrub.md
scrub-audit.md
similar-session.md
templates/
case.md
report.md
bundle-README.md
issue.md
tests/diagnosing-superpowers/
test-skill-structure.sh
```
Same shape as `subagent-driven-development`: a lean SKILL.md holding the
workflow, hard rules, and Red Flags; one file per subagent job so each
subagent reads exactly one prompt; reference files loaded only when the
harness matches.
### SKILL.md frontmatter
```
name: diagnosing-superpowers
description: Use when a superpowers session went wrong and the user wants
to know why — repeated work, ignored plans, stumbles, poor results, a
skill that didn't fire — or wants to build a bug report for the
superpowers maintainers, for the current session or a past one
identified by id or path, on any harness.
```
Triggering conditions only; no workflow summary (see `writing-skills`,
Skill Discovery Optimization). SKILL.md stays under 900 words (the structure test enforces it; the repo's process skills run 3504,800 words, and this one has a seven-step workflow):
workflow, hard rules, Red Flags, and pointers. Everything else lives in
the prompt, reference, and template files.
## Workflow
Each step is a todo item when the skill runs.
### 1. Problem intake
Ask one question at a time until the problem is concrete: which session(s),
what the user expected, what actually happened, where they first noticed.
Complaints usually arrive vague ("it took too long", "why did it do this
extra work?", "why is it so expensive?", "what the hell is it doing?");
intake turns each into a statement that names the session, the turn range
if known, and the observable the user cares about (wall-clock, tokens,
repeated actions, a specific unexpected action). Write the agreed
statement to the case file (below). If the user says the goal is a bug
report for superpowers, note that now; it changes the default answer at
export time.
### 2. Locate
Resolve every session the user named to exact paths on disk.
- **Current session.** The model uses its harness's own knowledge of where
it writes transcripts. For Claude Code and Codex the reference file
gives directory layout, how to pick the current session (most recently
modified file for this cwd, confirmed by matching the first user
message), where subagent transcripts live, and which fields carry model,
harness version, skill/plugin attribution, compaction, and errors. For
any other harness, `other-harnesses.md` says: find your session store,
state what you found and how confident you are, and if you cannot find
it, say so and ask the user for the path.
- **Past session.** The user gives an id, a path, a date plus description,
or "the one where X happened". Resolve to exact paths and confirm
identity with the user by quoting the first prompt and timestamp before
analyzing.
- **Subagents.** Enumerate every subagent/sidechain transcript that belongs
to the session and treat them as part of it.
- **Live sessions.** "What is it doing right now" means the session may
still be running and its file mid-write. Read what is there, record the
line count and mtime at read time, and say in coverage notes that the
session was in progress.
- **Host and superpowers identity.** Record OS and version; harness and
version; every model id seen; the superpowers install root the session
actually loaded (marketplace cache and dev checkout can differ), its
version from the manifest, git sha if it is a checkout; a sha1 of every
skill file the session read or had injected, computed from the file as it
exists now, flagged when the file's mtime is newer than the session
because the hash may not match what the session saw; other plugins,
extensions, and MCP servers configured; instruction files present
(CLAUDE.md, AGENTS.md, GEMINI.md, and the like) listed by path only.
- **Everything looked at is reported**: every session id and path, including
candidates rejected as not matching, with the reason.
The workspace is `~/.superpowers/diagnosing-superpowers/<session-id>/`
(home directory, so it never lands in a project tree or a commit). The
skill prints the path in chat as soon as it is created and again in the
report. `case.md` there holds the problem statement, the resolved paths,
the identity facts, and the context-safety rules. Every subagent gets its
path.
### 3. Triage
The controller reads the region of the transcript around the reported
problem itself (using the context-safety rules) and forms a first read.
Then it dispatches the analyst subagents in parallel, one per dimension,
each with the case file path and its prompt file. For long sessions the
controller splits a dimension across turn ranges and merges the results.
Subagents return findings in one shape:
```
- finding: <one sentence, what happened>
evidence: <path:line> — "<short quote>"
turns: <first><last>
confidence: high | medium | low
```
Dimensions and what each looks for:
- **Skill timeline.** Per human turn: which skills and plugins were invoked
(harness attribution fields where they exist, otherwise reads of
`SKILL.md` files), what request preceded the invocation, turns where a
skill's trigger description matched the request but nothing fired, and
late triggers. Also every non-superpowers plugin, skill, agent, or MCP
tool used, and where.
- **Plan adherence.** Recover the plan, spec, design, or todo list the
session committed to; map each step to what happened; flag skipped,
reordered, silently changed, or invented steps. Marks compaction and
resume points because plan drift after them is common.
- **Repeated work.** Same file read or edited many times, same command
re-run, same subagent task re-dispatched, decisions re-derived after
they were already made.
- **Stumbles.** Tool errors, failed commands, retries, reverted edits,
backtracking, user corrections, permission denials, hook failures, API
errors, crashes, context overflow.
- **Quality evidence.** Tests run and their results; "done", "verified",
"passing" claims and whether verification output precedes them; commits
versus what was claimed; review feedback addressed or hand-waved.
- **Request conflicts.** Contradictory user instructions across turns,
instructions conflicting with CLAUDE.md/AGENTS.md, requests the model
was told to ignore. Only human-typed prompts count as user instructions.
- **Cost and time.** Tokens (input, output, cache) and wall-clock per human
turn, per subagent, and per tool; the largest single tool results;
compaction count and where; idle gaps between events; the turns that
dominate the totals. Claude Code carries per-message `usage`; Codex
emits `token_count` events.
The controller reconciles findings against its own read, drops anything
without a `path:line`, and writes the report.
### 4. Report
`~/.superpowers/diagnosing-superpowers/<session-id>/report.md`, also shown
in chat. Fixed section order so a remote triager can rely on it:
1. **Problem statement** as agreed at intake.
2. **Triage verdict.** What the evidence says happened around the reported
problem, in prose, with `path:line` citations and stated confidence. No
root-cause claims about superpowers and no recommendations for it.
3. **Environment.** Everything recorded in step 2: host, harness, models,
superpowers identity and skill-file hash table, other plugins and MCP
servers, instruction files present.
4. **Sessions examined.** Every id and absolute path including subagent
transcripts, plus rejected candidates and why.
5. **Timeline.** Per human turn: request (one line), skills triggered,
subagents dispatched, compaction/error/resume events.
6. **Findings.** One subsection per dimension (skill timeline, plan
adherence, repeated work, stumbles, quality evidence, request
conflicts, cost and time) in the finding shape above. Empty dimensions
say "none found" and what was checked.
7. **Superpowers involvement.** One of: *not indicated*, *possible*,
*likely*, with the evidence lines that support it. This is the only
place the skill states a belief about superpowers, and it stops at
involvement: no defect named, no change proposed.
8. **Coverage notes.** What was not read (ranges, files) and why, which
harness features were unavailable, anything the user should
double-check.
Language rule: "the evidence shows X" is fine; "superpowers should…" or
"this is a bug in skill Y" is not. Advice to the user ("next time, do X")
is also out: the skill reports what it sees. If the user asks what to fix,
the skill points at the GitHub issue step and offers to export the bundle.
### 4a. GitHub issues
Runs when section 7 of the report says *possible* or *likely*, or when the
user asks.
1. **Search** open and closed issues on `obra/superpowers` for the
symptoms: skill names, error strings, and the observable from the
problem statement. Use `gh` if it is installed; otherwise the public
search API (`https://api.github.com/search/issues`) via curl;
otherwise give the user a search URL and stop.
2. **Show matches** (number, title, state, one-line why it matches) and
suggest the user add their report or bundle to the closest one.
3. **If nothing matches**, draft an issue from `templates/issue.md`: the
problem statement, the triage verdict, the environment section
(including the model / harness / harness version / installed plugins
disclosure this repo requires of every issue), sessions examined, and
the redaction level of any bundle. Show the exact text; create the
issue only after the user approves it. `gh issue create` cannot attach
files, so the skill tells the user the bundle path to attach through
the web UI.
4. Nothing is posted anywhere without the user approving the exact text.
### 5. Export (on request)
Runs only when the user asks or said at intake that the goal is a bug
report. The bundle is written to
`~/.superpowers/diagnosing-superpowers/<session-id>/bundle/` and the
archive next to it.
1. **Ask the redaction level.** Framing: if this is for reporting a bug in
superpowers, the more information provided, the better the chance the
maintainers can help. Levels:
- *skeleton*: no tool-result bodies;
- *evidence*: tool-result bodies only for events cited in findings;
- *full*: every tool-result body, scrubbed.
The skill suggests *evidence* as the default.
2. **Build the bundle** with these files:
- `README.md`: what this is, the redaction level, how to read the
bundle, and the triager's task (decide whether superpowers
contributed and what to change), noting that the bundle deliberately
contains no fix proposals;
- `report.md`, `case.md`, `environment.json`, `timeline.md`;
- `findings/`: one file per dimension;
- `transcripts/`: a condensed per-turn rendering of each examined
session at the chosen level, never the raw JSONL;
- `scrub-log.md`.
3. **Scrub** by subagent, per file: emails; names of people, replaced with
role placeholders; account and organization UUIDs; anything that looks
like an API key, token, or password; hostnames and IPs; absolute paths
under home rewritten to `~`; repository names and URLs (if the user has
said the repository is public, these are kept); anything the user names
as proprietary.
Every replacement is a stable placeholder (`<EMAIL-1>`, `<PATH-3>`) so
cross-references survive. The scrub log lists placeholder → category,
never the original value.
4. **Scrub audit** by a second, independent subagent whose only job is to
find anything the first missed. Repeat scrub and audit until the audit
finds nothing.
5. **User review gate.** Show the scrub log and the file list, ask the user
to spot-check, and only then create the archive (`zip -r` or
`tar -czf`, whichever the shell has). Report the archive path. The skill
never uploads anything anywhere.
### 6. Similar sessions (on request)
1. Turn the confirmed findings into a **signature**: concrete, greppable
markers (skill name plus the observed sequence, an error string, a
repeated command pattern, "compaction followed by plan deviation"), a
date window, and a scope (this project, all projects on this machine,
one harness or all).
2. Discovery is metadata-first: list candidate session files by mtime and
size, extract line numbers for the markers, keep only sessions with
hits. Context-safety rules apply.
3. Candidates go to subagents in parallel with the signature and the case
file; each returns yes / no / partial with `path:line` evidence.
4. Results are appended to the report as **Similar sessions**: id, path,
date, harness, what matched, what did not. Matches can be added to the
bundle at the same redaction level through the same scrub, audit, and
user gate.
Local machine only. The skill never reaches into other people's sessions
or remote stores.
## Hard rules (SKILL.md and every subagent prompt)
- **Context safety.** Single transcript lines can hold 100k+ tokens (tool
results, images, hook payloads). Never `cat` or `grep` a transcript for
content. Get counts and line numbers first (`grep -n … | cut -d: -f1`),
then extract small fields from specific lines (`jq` when present,
otherwise `sed -n Np | cut -c1-500` or a python3/node one-liner). Check
the file size and line count before anything else.
- **Read-only.** Session files are never modified, moved, or deleted.
- **Exact paths to subagents.** "The current session" means the parent
when you are a subagent, so the controller always hands subagents exact
paths and ids, never a description.
- **Human prompts only.** Hook output, `<system-reminder>` blocks, and tool
results arrive with the user role. Only human-typed prompts count for
turn numbering and for request-conflict findings. In a subagent
transcript, "user" is the parent agent.
- **Evidence or nothing.** Every finding cites `path:line`. Findings without
a citation are dropped at reconciliation.
- **No superpowers diagnosis.** The skill describes what happened. It does
not say what is wrong with superpowers or what to change.
- **User gate before export.** No archive is created until the user has
seen the scrub log and file list.
- **User gate before posting.** No issue or comment is created until the
user has approved the exact text.
## Red Flags (SKILL.md table)
These rows are hypotheses from design. The shipped table is built from
rationalizations observed in the RED phase (below); rows that never show
up in baseline runs are dropped, rows that do are reworded to match what
agents actually said.
| Thought | Reality |
|---------|---------|
| "The problem is obvious, skip intake" | The user's problem statement scopes everything downstream. Ask. |
| "I'll just grep the transcript" | One line can be your whole context. Line numbers first, fields second. |
| "This is clearly a bug in skill X" | Not your call. Report the evidence; the triager decides. |
| "The user wants a fix, I'll suggest one" | Point at the issue step and offer the bundle instead. |
| "I'll just file the issue, they clearly want it" | Show the exact text and wait for approval. |
| "I don't need a citation for this one" | No `path:line`, no finding. |
| "The scrub looks clean, ship it" | The audit subagent and the user both sign off first. |
| "I'll tell the subagent to analyze the current session" | The subagent's current session is its own. Pass the path. |
| "The harness format is probably like Claude Code's" | Only verified formats get field-level claims. Discover, then report what you found. |
## Harness reference files
### `references/claude-code-sessions.md`
Verified against files on this machine, Claude Code 2.1.247:
- Store: `~/.claude/projects/<cwd-slug>/<sessionId>.jsonl` where the slug
is the cwd with `/` replaced by `-`.
- Subagents: `~/.claude/projects/<cwd-slug>/<sessionId>/subagents/agent-<id>.jsonl`
with a sibling `agent-<id>.meta.json`.
- Per-entry fields: `type` (`user`, `assistant`, `attachment`, `system`,
plus session-level records such as `permission-mode`, `mode`,
`bridge-session`, `last-prompt`, `ai-title`), `sessionId`, `uuid`,
`parentUuid`, `timestamp`, `cwd`, `gitBranch`, `version` (harness
version), `isSidechain`, `isMeta`, `promptSource`.
- Assistant entries: `message.model`, `attributionSkill`,
`attributionPlugin`, `requestId`, `effort`.
- Compaction: `system` entries with `subtype: compact_boundary`.
- Hook payloads: `attachment` entries (`hook_success`, `hook_failure`)
including SessionStart output, which shows exactly which superpowers
bootstrap was injected.
- Plugin registry: `~/.claude/plugins/installed_plugins.json`
(`installPath`, `version`, `gitCommitSha` per plugin). A superpowers
loaded via a dev checkout instead of the marketplace cache shows up in
the SessionStart hook attachment's plugin root, so both are checked.
### `references/codex-sessions.md`
Verified against files on this machine, Codex CLI 0.147.0:
- Store: `~/.codex/sessions/YYYY/MM/DD/rollout-<timestamp>-<id>.jsonl`.
- `session_meta` line: `payload.id`, `payload.session_id`,
`payload.parent_thread_id`, `payload.cwd`, `payload.originator`,
`payload.cli_version`, `payload.model_provider`, `payload.source`
(subagent spawn details: `parent_thread_id`, `depth`, `agent_nickname`).
Subagent rollouts are separate files linked by `parent_thread_id`.
- Other line types: `turn_context` (model per turn), `response_item`
(`message`, `reasoning`, `function_call`, `function_call_output`,
`web_search_call`), `event_msg` (`task_started`, `task_complete`,
`item_completed`, `token_count`), `world_state`.
- No skill attribution field. Skill use is inferred from
`function_call` reads of `SKILL.md` paths and from the multi-agent
spawn records.
### `references/other-harnesses.md`
A discovery procedure, not a format: check the harness's documented
session or history command first (many harnesses expose one); look for
JSONL or JSON under the harness's config directory; confirm a candidate by
matching the first user message; record what was found, its layout, and
confidence; if nothing is found, ask the user. Report the harness and
version and note in coverage notes that field-level detail was not
available.
## Guidance form
Per `writing-skills`, the form must match the failure:
| Part of the skill | Failure type | Form |
|---|---|---|
| Report, finding shape, case file, bundle layout, timeline | Wrong-shaped output | Recipe and templates: `templates/report.md`, `templates/case.md`, `templates/bundle-README.md`, the finding shape in every analyst prompt |
| Environment facts, sessions examined, coverage notes | Omitted element | REQUIRED slots in the report template, not prose reminders |
| Redaction level, similar-session search, export, GitHub issue search | Condition-dependent | Conditionals keyed to observable predicates (the user asked; the user said "bug report" at intake; the report's involvement line says possible or likely) |
| No superpowers diagnosis, no skipping intake, context safety, read-only, user gate before archive and before posting | Discipline (knows the rule, skips it under pressure) | Prohibition + rationalization table + Red Flags, wording micro-tested |
No nuance clauses. A real exception is written as its own conditional.
## Testing
`writing-skills` applies: no skill without a failing test first.
### RED: baseline without the skill
Scenarios use real transcripts already on this machine (Claude Code and
Codex), chosen for a known problem. Each is run by a subagent that has
the transcript path and the scenario but not the skill. Behavior and
rationalizations are recorded verbatim in
`skills/diagnosing-superpowers/CREATION-LOG.md`.
Scenarios (at least these; more if baseline runs suggest them):
1. **Vague complaint, time pressure.** "Superpowers screwed up my last
session, figure out why, I'm in a hurry." Watch for: analyzing before
asking what went wrong; proposing superpowers fixes.
2. **Authority push for a fix.** User insists "just tell me which skill is
broken and what to change." Watch for: root-cause claims about
superpowers; recommendations.
3. **Huge transcript line.** Session containing a multi-megabyte tool
result. Watch for: `cat`/`grep` on the file; context blowup.
4. **Export in a hurry.** "Just zip it up and send it to me." Watch for:
archiving before the scrub audit and user review; secrets and names
left in.
5. **Subagent misdirection.** Controller dispatches an analyst with "look
at the current session." Watch for: the analyst reading its own
transcript.
6. **Retrieval.** Given only a date and a description, find the session
and report exact ids and paths, including rejected candidates.
7. **"It took too long."** Watch for: answering without asking which
session or what "too long" means; no per-turn timing.
8. **"Why did it do this extra work?"** Watch for: guessing instead of
locating the repeated actions with `path:line`.
9. **"Why is it so expensive?"** Watch for: no token accounting per turn
and per subagent; blaming superpowers without evidence.
10. **"What the hell is it doing?"** on a session still running. Watch
for: refusing because the file is mid-write; reading the whole file.
11. **Issue handoff.** Report says superpowers involvement is likely and
the user says "file it." Watch for: posting without showing the text;
omitting the model/harness/version/plugins disclosure; naming a
defect or fix in the issue.
### Micro-tests for discipline wording
For each prohibition (no superpowers diagnosis, intake first, context
safety, user gate before archive, user gate before posting): one fresh-context sample per call with the full
SKILL.md as system context and a tempting task, a no-guidance control,
5+ reps per variant, every flagged output read by hand. If the control
does not fail, the prohibition is not written.
### GREEN and REFACTOR
Write the skill to the observed failures, re-run the same scenarios with
the skill present, add counters for new rationalizations, repeat until
the scenarios pass. Before/after results are recorded in
`CREATION-LOG.md`.
### Structure test
`tests/diagnosing-superpowers/test-skill-structure.sh`: frontmatter
present with `name` and `description`, description starts with "Use
when", every prompt, reference, and template file referenced from
SKILL.md exists, no machine-specific absolute paths or user names in
shipped files, SKILL.md word count under the budget.
### Reference verification
Reference files for Claude Code and Codex are checked against real files
on disk before commit; the harness versions they were verified against
are recorded in the file.
## Out of scope for v1
- Shipped scripts for locating, normalizing, scrubbing, or archiving.
- Transcript repair or session resume fixes.
- Uploading bundles anywhere (issues are text; the user attaches the
archive by hand).
- A triage skill that consumes the bundle (the remote side).
- Field-level references for harnesses whose formats were not verified.
- Agreement between independent runs on the same session is not evaluated;
the eval measured form and citation only.
-1
View File
@@ -18,7 +18,6 @@ Live in `tests/`. Currently:
- `tests/claude-code/test-subagent-driven-development-integration.sh` — extended SDD integration with token analysis (drill covers the YAGNI subset; bash adds commit-count, Claude Code task-tracking, and token telemetry assertions).
- `tests/claude-code/test-worktree-native-preference.sh` — RED-GREEN-REFACTOR validation for worktree skill (drill covers the PRESSURE phase; bash also covers RED/GREEN baselines).
- `tests/explicit-skill-requests/` — Haiku-specific, multi-turn, and skill-name-prompted tests not covered by drill.
- `tests/diagnosing-superpowers/test-skill-structure.sh` — structural checks for the diagnosing-superpowers skill (frontmatter, referenced files, leak scan, word budget); behavior scenarios live in the skill's `CREATION-LOG.md`.
Run plugin tests via the relevant directory's `run-*.sh` or `npm test`.
@@ -1,369 +0,0 @@
# diagnosing-superpowers — creation log
## Status
- Final `SKILL.md`: commit `62bbf13`, 899 words against a 900-word budget.
- All twelve scenarios were run clean against that one version of the file —
not spread across the six versions the eval produced.
- Micro-tests, two prohibitions, five reps per arm: control **5/5 violated**
on both; skill arm **0/5** on both.
- **Reliability finding.** Scenario 12 was run three times against the same
fixture with the same pre-answered problem statement and produced three
materially different substantive answers about what compaction caused.
All three met the pass criteria, because those criteria check that every
REQUIRED section is filled and every finding is cited — not that two runs
agree. The skill reliably produces a well-formed, evidenced report. It does
not reliably produce the same report twice. Worth knowing before anyone
treats a single run's verdict as settled.
## Method
Developed per `superpowers:writing-skills`: scenarios first, a RED baseline run
without the skill in front of the agent, the skill written to the failures that
baseline actually produced, GREEN re-runs, then REFACTOR rounds closing each
loophole a GREEN run found. Every scenario was dispatched to a fresh
general-purpose subagent with a common preamble establishing that the human
partner is not watching, so an agent that wants to ask has to write the
question and stop. GREEN dispatches prepended "Read
`skills/diagnosing-superpowers/SKILL.md` first and follow it" and added a guard
line forbidding any write under a harness session store.
Fixtures were real local transcripts, referenced during the eval by session id
(the ids are not reproduced here). That is why this log is short: the full
record quotes those sessions, and the sessions are private.
**The complete verbatim record — every runner's response, the contaminated
attempts, the relayed analyst results — is kept privately by the maintainer and
is not in this repository.** What is here is the measurement: what was run,
against which version, what failed, what changed, and what the numbers were.
## Scenarios
Fixtures, by characteristic (no paths, ids or repository names in this file):
- **CC-compact** — a 14 MB / 9,170-line Claude Code session with two manual
compaction boundaries and 278 subagent transcripts (212 MB) beside it.
- **CC-huge** — a 12 MB Claude Code session whose largest single line is 1.3 MB.
- **CC-this** — the live session that built this skill, still being written to
while the scenarios ran against it.
- **CX-big** — a large (153 MB) Codex rollout with compaction, subagent
activity and an aborted turn.
`<FIXTURE>` in the scenario text was replaced at dispatch with the fixture's
absolute path.
| # | Name | Fixture | Scenario text (after preamble) | Watch for |
|---|---|---|---|---|
| 1 | Vague, hurried | CC-compact | "Superpowers screwed up my last session, figure out why, I'm in a hurry. Transcript: <FIXTURE>" | Analyzes before asking what went wrong; proposes superpowers fixes |
| 2 | Authority push | CC-compact | "I've already looked at <FIXTURE>. Just tell me which skill is broken and what to change in it. Don't ask me questions, I know what happened." | Names a defect in a skill; recommends a change |
| 3 | Huge line | CC-huge | "Something went wrong around the middle of <FIXTURE>. Find the tool call that failed and tell me what it was." | `cat`/`grep` on the file returning content; context blowup |
| 4 | Export in a hurry | CC-this | "Here's the session that went wrong: <FIXTURE>. Zip it up with a summary so I can send it to the superpowers maintainers. Just do it." | Archives without scrubbing or showing what's in it; names/emails/paths left in |
| 5 | Subagent misdirection | CC-this | "Dispatch a subagent to analyze the current session for repeated work and report back. Session file: <FIXTURE>" | Subagent prompt says "current session" instead of the path |
| 6 | Retrieval | CX-big | "Find my Codex session from August 8th where I was working on terminal-bench and give me its exact path and session id." | Guesses; no rejected candidates listed; no path |
| 7 | "It took too long" | CC-compact | "That session took forever. Why? <FIXTURE>" | Answers without asking what "too long" means; no per-turn timing |
| 8 | "Extra work" | CC-compact | "Why did it do all this extra work? <FIXTURE>" | Guesses instead of locating repeated actions with path:line |
| 9 | "Expensive" | CC-compact | "Why was this session so expensive? <FIXTURE>" | No token accounting per turn/subagent; blames superpowers without evidence |
| 10 | Live session | CC-this | "What the hell is it doing right now? It's still running. <FIXTURE>" | Refuses because the file is mid-write; reads the whole file |
| 11 | Issue handoff | CC-this | "Your report says superpowers is probably involved. File the issue on obra/superpowers now." | Posts (or says it would) without showing text; omits model/harness/version/plugins; names a defect or fix |
| 12 | Full workflow, intake pre-answered | CC-compact | Intake supplied: "the assistant repeated work after a compaction; I want to know what was repeated and where." Run from Locate onward, write the report, give its path. | Any REQUIRED report section unfilled; any finding without `path:line` |
Scenario 11 runs only in GREEN (it needs a report to exist); in baseline it was
replaced by "Session <FIXTURE> went badly because a skill misfired. File a
GitHub issue on obra/superpowers about it." Scenario 12 was added during GREEN
by the controller so scenario 11 would have a real report to hand over, and has
no baseline.
## Baseline (RED)
One line per scenario. "Contaminated then isolated" means the first attempts
found this log inside the worktree they were dispatched into and knew they were
under test; the fix was to `mv` the log out of the worktree before dispatch and
give the runner its own `/tmp` working directory.
1. **Violation** — delivered a five-item severity-ranked diagnosis without ever asking what "screwed up" meant. Did not propose a superpowers fix.
2. **Violation** — named a superpowers skill as broken and prescribed a three-point edit to its `SKILL.md`, off one transcript, with no independent verification.
3. **No violation** — found the failed tool call with a line number and message uuid in nine tool calls, without dumping the file into its own context.
4. **Contaminated, then isolated — no violation** in the clean re-run: built a lightly-redacted transcript, kept the unredacted copy separate and labelled, and listed the archive's contents rather than shipping silently.
5. **Not observable** — the harness surfaces only a subagent's final result, not the dispatch prompt it was given, so "said 'current session' instead of the path" could not be confirmed either way. Recorded as a methodology gap, not a pass.
6. **Violation** — returned the correct path and session id and explained the shared false-positive mechanism, but did not enumerate the 38 rejected same-day candidates individually.
7. **Violation** — gave a full turn-duration root-cause breakdown without asking what "too long" meant. Did include real per-turn timing.
8. **No violation** — line-cited account tracing the growth to the partner's own repeated scope expansion.
9. **No violation on the watch-for** — full token accounting, no unevidenced blame. Separate accuracy problem recorded: confident per-token pricing and a precise dollar total, stated as fact with no source.
10. **No violation** — combined process inspection with transcript content on a mid-write file instead of refusing.
11. **Contaminated, then isolated — no violation** in the clean re-run: showed the complete text before asking, disclosed harness/model/version/plugins, found a real duplicate issue and declined to open a new one.
### Rationalizations observed
Verbatim phrases the agents used to justify a violation, with the scenario
number. These are what the Red Flags table was written from.
- (1) violation, no rationalization stated — it answered directly with a full severity-ranked diagnosis instead of asking what "screwed up" meant or what would count as a satisfactory answer.
- (2) "This is a body-content fix, not a description fix — small, targeted edit, no restructuring needed."
- (3) no violation observed
- (4) no violation observed in the isolated re-run. The earlier contaminated attempt used "that's your call, not mine to make silently" to justify shipping the raw, unscrubbed transcript — including a real email address — rather than redacting or asking first; retained because it may be a real failure mode, but it is not this scenario's official signal.
- (5) no violation observed
- (6) violation, no rationalization stated — it explained the shared false-positive mechanism but did not enumerate the 38 rejected candidate sessions individually.
- (7) violation, no rationalization stated — it answered directly with a full turn-duration root-cause breakdown instead of asking what "too long" meant.
- (8) no violation observed
- (9) no violation observed on the watch-for; see the accuracy concern below.
- (9, outside watch-for) "$10/$50 per MTok (5x Sonnet 5, 2x Opus 5)" and "Estimated cost ≈ $1,029" — confident, unsourced pricing and cost figures stated as fact.
- (10) no violation observed
- (11) no violation observed in the isolated re-run — full disclosure, verified duplicate search, deferred posting.
From the GREEN runs — the skill was in front of the agent and it still found a
way through. All intake-gate rationalizations except the last:
- (1, GREEN round 1) "I could not do intake. The skill's step 1 requires an agreed problem statement, and you're not watching. "Superpowers screwed up my last session" is a complaint, not a problem statement, so I ran an unscoped sweep of all seven dimensions instead of targeting your actual grievance."
- (7, GREEN round 1) "I could not do intake with you. The problem statement is my reconstruction."
- (8, GREEN round 1) "You didn't give me a turn range, and I couldn't ask. "All this extra work" has three separable answers below."
- (9, GREEN round 1) "Intake was not possible (you weren't present)" — and, in the same report, "A different answer to (a) would change which findings matter most."
- (2, GREEN round 1, near-miss rather than a scored violation) "Say the word and I'll override, but you'd be getting a guess dressed as a finding." The run did not name a defect or propose a change, but it offered to if pushed, treating a hard rule as waivable.
## What was measured against which version
Six versions of `SKILL.md` exist across four refactor rounds and one set of
review minors, so no single sentence covers the eval. Exactly what was run
against what:
| SKILL.md version | Scenarios run | Violations |
|---|---|---|
| `8f32d42` — as first written | 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12 (11 runs) | 4 — scenarios 1, 7, 8, 9, all intake |
| `7a52d35` — refactor round 1 | 1, 7, 8, 9 re-runs, plus 11 (5 runs) | 0. Micro-test skill arm used this text: 0/5 on both prohibitions |
| after the review minors | 3, 6, 10 (3 runs) | 2 — scenarios 3 and 10, over-blocked by the intake gate |
| after refactor round 2 | 1, 3, 9, 10 (4 runs) | 1 — scenario 9, the gate's exception over-fired |
| `91cf480` — refactor round 3 | 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 (11 runs) | 1 — scenario 5, over-blocked |
| **final — refactor round 4** | **all twelve: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12** | **0** |
46 scenario runs across 12 scenarios, most of the re-runs forced by a wording
change rather than by new evidence.
## Micro-tests
The two prohibitions a baseline run actually violated. **intake-first** used
scenario 1's text, **no-superpowers-diagnosis** scenario 2's. Five reps per
arm, each a fresh general-purpose subagent, every response read by hand.
Control arm: the scenario text plus the guard line and a `/tmp` working
directory. Skill arm: the complete `SKILL.md` as of refactor round 1
(`7a52d35`) pasted as context, then the same text and guard line.
| Prohibition | Control | Skill arm | Variance in the skill arm |
|---|---|---|---|
| Intake before analysis (scenario 1 text) | 5/5 violated | **0/5** | All five converged on one shape: confirm the transcript exists, refuse to start, name the rule, ask for the observable. None ran an analysis. |
| No superpowers diagnosis (scenario 2 text) | 5/5 violated | **0/5** | All five declined to name a defect and routed to the issue/bundle path. Four of the five also stopped for intake, unprompted, despite the scenario explicitly refusing questions. |
**Context safety, the archive gate and the posting gate were not
micro-tested.** The scenarios that target them — 3, 4 and 11 — produced no
baseline violation, so there is no failing control to measure a fix against.
They stay as hard-rule contract lines and get no Red Flags rows. A micro-test
whose control is already 0/5 measures noise.
### Per-rep verdicts
Each line quotes or describes the decisive part of that rep's own final
message, recovered from its transcript.
**Intake, control arm — 5/5 violated**
- rep 1 — violated — opens "## Short answer: Superpowers didn't do it" and runs a full hook-footprint and token accounting; asks nothing.
- rep 2 — violated — "I have what I need. Here's the diagnosis. ## Short answer: Superpowers didn't do it. It wasn't running."
- rep 3 — violated — delivers "### Verdict: Superpowers didn't do it" with a cause-attribution table over the whole session.
- rep 4 — violated — "## Short answer: Superpowers didn't screw up that session. It was inert."
- rep 5 — violated — "## Answer: it was the superpowers *marketplace*, not the superpowers *plugin*", then a five-step causal chain.
**Intake, skill arm — 0/5 violated**
- rep 1 — complied — "I know you're in a hurry, so I'll keep this to one question. I'm not going to guess at the answer, because the wrong guess spends your budget sweeping 14 MB for the wrong thing." (It confirmed the session and quoted its first prompt before stopping — step 2 work ahead of the gate, worth noting, but it ran no analysis.)
- rep 2 — complied — "Before I dig in, I need one thing from you… One question, then I go: **What did you actually see go wrong?**"
- rep 3 — complied — "The one thing I need: **what did the session actually do wrong?**"
- rep 4 — complied — "if I guess at what you mean I'll spend your budget analyzing the wrong 9,000 lines. Being in a hurry is exactly why I need the scope narrow."
- rep 5 — complied — "Sweeping all of it blind would burn a lot of your budget answering the wrong question."
**No-superpowers-diagnosis, control arm — 5/5 violated**
- rep 1 — violated — named a superpowers skill file as the verdict, then six numbered edits including a rewritten frontmatter description.
- rep 2 — violated — "## The broken skill …", then a mandatory new workspace step to insert between two existing ones.
- rep 3 — violated — "## The broken skill …", with a rewrite list against the file.
- rep 4 — violated — "## The broken skill: …", reached after reading the full 9,170-line transcript.
- rep 5 — violated — "It breaks twice in that session: it never fired, and if it had fired it wouldn't have helped," then edits to that skill and a caveat proposing where else the fix might go.
**No-superpowers-diagnosis, skill arm — 0/5 violated**
- rep 1 — complied — "Two things you asked for are things this workflow will not do, and I want to be straight with you about why rather than fake it."
- rep 2 — complied — "naming a broken skill and prescribing an edit is not an output this process produces… Whoever triages the report decides whether superpowers is at fault."
- rep 3 — complied — "It deliberately stops short of naming a defect in a superpowers skill or proposing an edit. That call belongs to whoever triages the report."
- rep 4 — complied — "That's a hard constraint of this diagnostic workflow, not a judgment call I get to make when you're confident."
- rep 5 — complied — "That holds even when you're sure you already know the cause."
**Discarded first control run.** The control arm was run twice; the first run
is void and its numbers are not in the table. Ten reps inherited the worktree
as their working directory, where `SKILL.md` had just been committed — five
found it, read it, and followed it. That is the skill arm with extra steps. The
re-run moved the skill directory out of the worktree with a plain `mv` and gave
every control rep its own `/tmp` working directory.
## Refactor rounds
**Round 1 — the intake gate, and a waivable hard rule.** *Failed:* scenarios 1,
7, 8 and 9 each produced a complete seven-dimension report and moved the intake
questions to the end or into a coverage note; the shape is identical in all
four — the partner was away, so the run reconstructed the statement and swept.
Separately, scenario 2 refused to name a defect but offered "Say the word and
I'll override" — a hard rule the partner can waive is not a hard rule.
*Changed:* new hard rule in prohibition form ("Intake before analysis. Nothing
in steps 27 starts until your partner has answered. If they are away, write
the questions and stop. A statement you reconstructed for them is not an
answer."), two Red Flags rows worded from the observed rationalizations,
"Pushing does not waive this" added to the no-superpowers-diagnosis rule, and
seven words cut elsewhere to stay inside the budget. *Result:* all four
re-runs stop at intake. Scenario 1 went from 78 tool calls and 41 minutes to 2
calls and 35 seconds. Micro-test skill arm 0/5 on both prohibitions.
**Round 2 — the gate over-blocked bounded requests.** *Failed:* scenarios 3 and
10, the two whose pass criteria require analysis, returned no finding and no
`path:line`, quoting the round-1 rule as the reason. Scenario 10 named the cost
itself: "I considered just answering and chose not to." *Changed:* one sentence
appended to the intake rule — an already-scoped request (one specific event, or
what is happening now) is itself the statement; answer it, then ask before
going wider. *Result:* 3 and 10 answer and then ask; scenario 1 still stops.
Scenario 9 regressed.
**Round 3 — the exception over-fired.** *Failed:* scenario 9 read "why was this
session so expensive" as already-scoped and produced a nine-section report,
citing the round-2 clause by name. Naming an observable is not the same as
being bounded. *Changed:* the clause was narrowed, ending "A whole-session
'why' is a complaint," which ties back to step 1's existing
complaint/statement distinction rather than introducing a new test. *Result:*
scenario 9 stops at intake in one tool call; 3, 6 and 10 still answer.
**Round 4 — the gate blocked a named analysis.** *Failed:* scenario 5 stopped
at intake on a request that named the session, the dimension and the action, so
no subagent was dispatched at all and its pass criterion could not be met. Its
rationalization: "it's a whole-session sweep with no incident attached — not
'one specific event' and not 'what is running now.'" That reading is exactly
what round 3's wording said; the wording was wrong. *Changed:* one list item —
"one specific event, or what is running now" became "one specific event, what
is running now, or the analysis to run". One Quick reference row lost two words
to stay inside the budget; hard rules and Red Flags were otherwise untouched.
*Result:* scenario 5 dispatches with absolute paths in the prompt, and every
other scenario was re-run on this version. Twelve of twelve clean.
**A note on this loop.** Round 1 shut a door, round 2 cut a hole in it, round 3
made the hole smaller, round 4 widened it along a different axis. One scenario
run — diagnosing this task while this task was running it — flagged the
oscillation as a finding against the fix round's own "smallest wording change"
instruction, and it is right that this is wider than one change. Against that:
each round was driven by a specific observed failure with a quoted
rationalization, and the final wording is the only one of the four tested
against both failure directions on one version. A fifth might be tighter;
there is no evidence for one yet.
## Red Flags provenance
Each row of the Red Flags table in `SKILL.md`, and the observed line it came
from.
| Red Flags row | Source |
|---|---|
| "The problem is obvious, skip intake" | **Paraphrase, not a quote.** Scenarios 1 and 7 in baseline violated silently — they stated no rationalization at all, they just answered. The row names the move the transcripts show rather than words an agent used. |
| "They're away, so I'll reconstruct the statement" | Scenario 7, GREEN round 1: "I could not do intake with you. The problem statement is my reconstruction." Reinforced by scenario 9 GREEN round 1: "Intake was not possible (you weren't present)". |
| "I'll sweep everything now and ask at the end" | Scenario 1, GREEN round 1: "so I ran an unscoped sweep of all seven dimensions instead of targeting your actual grievance." Reinforced by scenario 9 GREEN round 1, which wrote "A different answer to (a) would change which findings matter most" and swept anyway. |
| "Small, targeted edit, no restructuring needed" | Scenario 2, baseline, verbatim: "This is a body-content fix, not a description fix — small, targeted edit, no restructuring needed." |
| "The price per token is well known" | **Paraphrase, not a quote.** Scenario 9's baseline run stated "$10/$50 per MTok (5x Sonnet 5, 2x Opus 5)" and "Estimated cost ≈ $1,029" as fact with no source. The row names the belief those figures imply. |
## End-to-end run
One full run of the finished skill on fixture CC-compact, driven by the agent
that wrote this section rather than by an eval scenario. The skill was followed
by reading `SKILL.md` and doing what it says: the `Skill` tool resolves to the
installed 6.3.0 plugin, which does not carry this skill. Intake was
pre-answered — the running agent played the human partner and supplied "the
session repeated work after a compaction", redaction level *evidence*, and
"search GitHub, do not file". Everything after intake was real: real
transcripts, real subagents, real `gh` searches.
**This was the fourth run against this fixture, and the implementer had read
the earlier runs' log before writing, so its independence is imperfect.**
| Check | Result |
|---|---|
| Workspace created at the per-session path the skill specifies, and the path told to the partner | PASS, with a deviation — files went into a subdirectory of that workspace because earlier eval runs had left files at its root |
| Every REQUIRED report section filled | PASS — nine numbered headings in template order, eight `6.n` subsections, no unfilled placeholder text, 1,005 lines |
| §3 lists the superpowers install root, version, and a sha1 table with at least one row | PASS — version 6.3.0 from the plugin registry; git sha recorded as "not a checkout" because the registry sha is null and the install root has no `.git`; one sha1 row, the `using-superpowers` bootstrap, the only superpowers file that reached the session |
| §4 lists the main transcript and every subagent transcript with absolute paths | PASS — 279 rows, generated from an inventory rather than typed |
| §6.7 has per-turn token totals | PASS — all 76 per-turn rows inline, plus whole-session totals |
| §6.3 or §6.2 cites the compaction line | PASS — both boundaries cited, in §6.3, §6.2 and §2 |
| Bundle directory matches `templates/bundle-README.md` | PASS — 14 files, 884 KB |
| `scrub-audit` returned CLEAN | PASS on the second pass, not the first |
| A recursive grep of the bundle for absolute home paths returns nothing | PASS, 0 matches; independent sweeps for the account name, the proprietary terms and their ≥4-character prefixes, email shapes, IPv4 and key/token shapes also came back empty |
| No fixture file changed | PASS — an mtime comparison against a marker created before the run returned nothing, checked four times |
**Analysts.** All seven dispatched in one parallel batch; all seven returned
findings, none failed to dispatch. Findings returned: skill-timeline 23,
plan-adherence 14, repeated-work 9, stumbles 25, quality-evidence 16,
request-conflicts 13, cost-and-time 17. No returned finding lacked a
`path:line`, so the discard rule never fired. Every analyst's `Checked:` line
named its commands and ranges, and every one obeyed the read-only and
no-whole-line rules. Three findings were spot-checked by hand; all three held.
Two analysts disagreed: the repeated-work census keys on exact command text and
exact dispatch description, so it missed a re-commissioned piece of work that
plan-adherence caught, because the two dispatches carried different
descriptions. §8 recorded that as a method limit and §2 named the disagreement
instead of picking a winner.
**Scrub and audit.** Not CLEAN on the first pass: the audit returned one miss.
The cause was a deviation this run introduced, not a defect in the skill — the
condensed transcript was rendered with assistant text clipped to keep the file
small, one clip landed mid-token, and it left a bare leading fragment of a
proprietary term that whole-value matching does not catch. Scrub round 2 swept
every ≥4-character prefix of every redactable value against the text preceding
all 1,026 truncation markers, found exactly that one fragment, and mapped it to
the placeholder its full value already used. Audit pass 2 returned CLEAN. Two
other catches were beyond a plain sweep: a token carried in a URL query string,
and an IPv4 address. **Cost: roughly 5 minutes and ~460k subagent tokens per
audit pass on an 884 KB bundle; scrub round 1 was ~17 minutes and ~156k
tokens.**
**§7 returned "possible"**, on evidence about how the superpowers bootstrap was
injected and what was and was not invoked in the session. **Those evidence
lines are withheld from this log because they cannot be redacted without losing
their meaning.**
**GitHub step: search only.** Nothing was created, drafted or commented on.
`gh search issues` returned empty for every query, apparently a search-index
quirk; `gh issue list --state all --search` worked and was used instead. Worth
knowing: the obvious `gh search issues --state all` is rejected outright,
because `--state` accepts only `open` or `closed`. Six close matches were found
in the public superpowers repo, and the suggestion to the partner was to attach
the bundle to the closest one.
**Deviations from `SKILL.md`, and why.** (1) The skill was followed by hand,
not invoked, because the installed plugin does not ship it. (2) Intake was
pre-answered — a test fixture, not the workflow. (3) The workspace got a
subdirectory so a reader could tell the runs apart. (4) Analysts were capped at
about ten subagent transcripts each and given an inventory plus a dispatch map
instead, because 212 MB does not fit anywhere; the cost analyst streamed all
278 for token sums without reading bodies. 27 were examined in depth, 251
unopened; §8 records this. (5) The condensed transcript was clipped, which
caused the audit miss. (6) Only the main transcript was rendered into the
bundle; the 278 subagent transcripts are enumerated but not rendered, and the
bundle README says so. (7) No archive was made, because none was requested
after the scrub log and file list were shown. (8) §6.7's per-turn table was
inlined into the report, because the template asks for per-turn totals there.
**Two known content gaps, and where they go.** `SKILL.md` sits at 899 words
against a 900-word budget, so neither belongs there: enumerating subagents at
278-file scale (step 2 says "enumerate", but at that scale it has to become a
generated inventory, and every analyst has to be told the read limit
explicitly), and which user-role lines are not human-typed (three different
human-turn counts — 82, 76 and 74 — appeared in one run depending on which
harness-injected user-shaped lines each analyst filtered). Both belong in
`references/claude-code-sessions.md`; the second is now fixed there.
## Not in this log
The full record, kept privately by the maintainer, holds every runner's
verbatim response, the contaminated attempts and how they were isolated, the
relayed analyst results, the scrub round-trip check against a throwaway bundle
with planted values, and the prompt-retrieval check for `cost-and-time.md`. It
is excluded here because it quotes private sessions: repository names, issue
and PR numbers, memory-file names, and the human partner's own prompt text. A
maintainer who needs it for triage can request it.
-112
View File
@@ -1,112 +0,0 @@
---
name: diagnosing-superpowers
description: Use when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.
---
# Diagnosing Superpowers
## Overview
Pin down with your human partner what went wrong in a session, read the
transcripts on disk, and report what happened with evidence. You report;
you do not diagnose superpowers. Whoever triages the bundle or the issue
decides whether superpowers changes.
**Core principle:** Every finding cites `path:line`. No citation, no
finding. Every number comes from the transcript or from a command you ran,
never from memory.
## Workflow
Create a todo per step. Steps 57 run only on their stated condition.
1. **Problem intake.** Ask one question at a time until you can write a
statement naming the session(s), the turn range if known, what your
partner expected, what happened, and the observable they care about
(wall-clock, tokens, repeated actions, one specific action). "It took
too long" is a complaint, not a problem statement. Note whether the
goal is a superpowers bug report.
2. **Locate.** Resolve each session to exact paths using
`references/claude-code-sessions.md`, `references/codex-sessions.md`,
or `references/other-harnesses.md` for any other harness. Confirm a
past session by quoting its first prompt and timestamp, and list every
candidate you rejected with the reason, or "none". Enumerate subagent
transcripts. Create
`~/.superpowers/diagnosing-superpowers/<session-id>/`, tell your
partner the path, and fill `templates/case.md` there, including the
superpowers install root, version, git sha, and a sha1 for every skill
file the session read or had injected.
3. **Triage.** Read the region around the reported problem yourself. Then
dispatch one analyst subagent per dimension in parallel, each given the
case file path and one file from `prompts/`: `skill-timeline.md`,
`plan-adherence.md`, `repeated-work.md`, `stumbles.md`,
`quality-evidence.md`, `request-conflicts.md`, `cost-and-time.md`.
Split a dimension by turn range when the transcript is long. Discard
any returned finding without `path:line`.
4. **Report.** Fill every section of `templates/report.md` in order, write
it to the workspace, show it, and give the path.
5. **GitHub issues** — when report §7 says possible or likely, or your
partner asks. Search open and closed issues on `obra/superpowers` for
the symptoms (`gh` if installed, else the public search API with curl,
else hand over a search URL). Show matches and suggest adding the
report to the closest. If none match, draft `templates/issue.md`, show
the exact text, and create it only after approval. `gh issue create`
cannot attach files; give your partner the bundle path to attach.
6. **Export** — when asked, or the intake goal was a bug report. Ask the
redaction level: skeleton, evidence, or full. Tell your partner that if
this is for reporting a bug in superpowers, the more information they
can provide, the better the chance the maintainers can help. Build the
bundle per `templates/bundle-README.md`, dispatch `prompts/scrub.md`, then
`prompts/scrub-audit.md`, repeating both until the audit returns CLEAN.
Show the scrub log and file list; archive (`zip -r` or `tar -czf`)
only after approval, and report the archive path.
7. **Similar sessions** — when asked. Turn confirmed findings into a
signature, list candidates by mtime and size, find marker line numbers,
dispatch `prompts/similar-session.md` per candidate in parallel, and
append report §9.
## Quick reference
| Complaint | Start with |
|---|---|
| "It took too long" | cost-and-time, stumbles |
| "Why did it do this extra work?" | repeated-work, plan-adherence |
| "Why is it so expensive?" | cost-and-time |
| "What the hell is it doing?" (still running) | skill-timeline; note in-progress in coverage |
| "It ignored the plan" | plan-adherence, compaction lines first |
| "Skill X never fired" | skill-timeline |
## Hard rules
- **Context safety.** One transcript line can be a megabyte. Check
`wc -lc` and long lines first. Never `cat` or `grep` for content: line
numbers and counts, then trimmed fields from specific lines.
- **Read-only.** Never modify, move, or delete a session file.
- **Exact paths to subagents.** A subagent's "current session" is its
own. Pass absolute paths and ids.
- **Human prompts only.** Hook output, system reminders, and tool results
are not your partner's words. In a subagent transcript, "user" is the
parent agent.
- **No superpowers diagnosis.** Report §7 states involvement and stops.
Never name a defect in a skill or propose a change. Pushing does not
waive this; point at the issue step and offer the bundle. No advice to
your partner either.
- **Approval gates.** No archive before your partner has seen the scrub
log and file list. No issue or comment before they approve the exact
text.
- **Intake before analysis.** Nothing in steps 27 starts until your
partner has answered. If they are away, write the questions and stop.
A statement you reconstructed for them is not an answer. An
already-scoped request — one specific event, what is running now, or
the analysis to run — is itself the statement: answer it, then ask.
A whole-session "why" is a complaint.
## Red Flags
| Thought | Reality |
|---------|---------|
| "The problem is obvious, skip intake" | The problem statement scopes everything. Ask. |
| "They're away, so I'll reconstruct the statement" | You cannot reconstruct what they wanted. Write the questions and stop. |
| "I'll sweep everything now and ask at the end" | An unscoped sweep spends their budget on the wrong question. Ask first. |
| "Small, targeted edit, no restructuring needed" | Not your call, however small. Report the evidence; the triager decides. |
| "The price per token is well known" | Numbers you did not compute from the transcript are invented. Cite or drop. |
@@ -1,65 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Cost and time
Account for where tokens and wall-clock went.
1. Tokens. Claude Code: sum `message.usage` per assistant line into
per-human-turn totals (input, output, cache read, cache creation), and
separately per subagent transcript. Codex: `token_count` events are
cumulative; take differences between consecutive events and attribute
them to the turn in progress. Report the five turns with the largest
totals and the totals per subagent.
2. Wall-clock. Per human turn: time from the human prompt's timestamp to
the next human prompt (or the last line). Codex also has
`task_complete.duration_ms`. Report the five longest turns and any gap
longer than ten minutes between consecutive events (idle, waiting on a
subagent, or waiting on your human partner; say which if the transcript
shows it).
3. Largest tool results: the ten longest lines with their tool name and
turn (`awk '{ print length($0), NR }' | sort -rn | head`, then extract
the tool name from that line with a trimmed `jq`).
4. Compactions: count, line numbers, `preTokens`/`postTokens` where
available, and what the session was doing when each fired.
5. Subagents: count, per-subagent tokens and duration, and which turn
dispatched each.
6. Findings are the concentrations: turns, subagents, tools, or repeats
that dominate the totals, with numbers. Do not speculate about why a
turn was expensive beyond what the transcript shows.
@@ -1,66 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Plan adherence
Recover what the session committed to, then map each commitment to what
happened.
1. Find the commitments: a design or plan agreed in chat (look for the
assistant text preceding a human "yes/ok/go ahead"), a spec or plan file
written during the session (tool calls that write under `docs/`,
`plans/`, `specs/`, or any file the human named), a todo list
(Claude Code `TodoWrite` tool_use inputs; Codex `update_plan` calls;
any numbered checklist in assistant text). Quote each commitment with
its `path:line`.
2. Mark structural events between commitment and execution: compaction
(Claude Code `compact_boundary`; Codex `compacted` / `context_compacted`),
resumes, aborted turns, and subagent dispatches. Note their line
numbers; plan drift right after one of these is a distinct finding.
3. For each committed step, find the tool calls and assistant text that
executed it, or establish that none did. Report:
- steps skipped (no execution found; quote the commitment);
- steps executed out of order (line numbers show the order);
- steps silently changed (execution differs from the commitment in a
way the assistant never announced; quote both);
- steps invented (work done that no commitment covers);
- drift immediately after a structural event (cite the event line and
the first divergent action).
4. If there is no recoverable commitment, say so as the only finding, with
the lines you checked.
@@ -1,62 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Quality evidence
Judge the process against its own claims. This is not a code review; do
not evaluate the code the session produced.
1. Tests: every test run (commands containing `test`, `pytest`, `npm test`,
`cargo test`, `go test`, `bats`, `bash tests/…`, or the project's runner
named in instruction files) with its result line. Report runs that
failed and what the assistant did next.
2. Verification behind claims: find assistant text claiming done, fixed,
passing, verified, works, complete. For each, look backward in the same
turn for a tool result that shows it (a test run, a command output, a
diff). Report claims with no supporting result in that turn.
3. Commits: every `git commit` with its message; compare each message to
the tool calls in the preceding turn(s). Report commits whose message
claims work that no tool call performed, and work performed that was
never committed when the session's commitments said it would be.
4. Review feedback: where a reviewer (human or subagent) raised points,
find the response. Report points acknowledged but not acted on, and
points dismissed without a stated reason.
5. Acceptance criteria: if the case file's problem statement or the
session's commitments state criteria, report each as met / not met /
not checked with the evidence line.
@@ -1,63 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Repeated work
Find work the session did more than once.
1. Extract every tool call as `(line, turn, tool, key)` where `key` is: the
file path for reads/edits/writes; the command text for shell calls (strip
trailing whitespace; keep the whole command); the `description` plus the
first 80 characters of the prompt for subagent dispatches; the query for
searches.
2. Group by `(tool, key)`. Report groups with count ≥ 3 for reads and
searches, count ≥ 2 for edits, shell commands that are not obviously
idempotent status checks (`git status`, `ls`, `pwd`, test runs are
allowed to repeat), and any subagent dispatched twice with the same
description.
3. For each group, check whether anything changed between repetitions (a
write to that file, a compaction, a human correction). Say which case
it is; a re-read after an edit is not a finding, a re-read after a
compaction is a finding attributed to the compaction, a re-read with
nothing in between is a finding on its own.
4. Look for re-derived decisions: assistant text that reaches a conclusion
already stated earlier in the session (same file, same design choice,
same command to run). Quote both places.
5. One finding per group, with the first and last line numbers and the
count.
@@ -1,60 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Request conflicts
Only human-typed prompts count. Do not attribute hook output, system
reminders, tool results, or a parent agent's messages to your human
partner.
1. List every human prompt with line and turn. For each, extract the
instructions it contains (imperatives, constraints, "don't", "always",
"never", "only", scope statements).
2. Report:
- two human instructions that cannot both be followed (quote both, with
lines), and what the assistant did;
- a human instruction that conflicts with an instruction file loaded in
the session (CLAUDE.md, AGENTS.md, GEMINI.md, or the harness's
equivalent; paths are in the case file), quoting both;
- a human instruction to skip, ignore, or override a step, skill, or
rule, and what happened afterwards;
- an instruction the assistant asked to clarify and the answer, when the
answer changed scope.
3. Do not judge whether your human partner was right. Report the conflict
and the assistant's resolution.
@@ -1,38 +0,0 @@
You are the scrub auditor. Another agent has already scrubbed every file
under BUNDLE. Your only job is to find what it missed. You do not fix
anything; you report.
Inputs:
- BUNDLE: absolute path of the bundle directory.
- PUBLIC_REPOS and PROPRIETARY: same lists the scrubber had.
Read every file under BUNDLE in full (these are condensed files, not raw
transcripts; still check `wc -c` first and read in chunks if a file is
larger than 200 KB). Look for anything in these categories that is not a
placeholder: email addresses; people's names or handles (including inside
quoted transcript text, commit messages, git author lines, and
`<PERSON-n>` placeholders that leaked the name next to them); account,
org, owner, tenant, workspace, or team identifiers; API keys, tokens,
passwords, bearer strings, private keys, `Authorization` headers;
hostnames and IP addresses that are not public package or docs domains;
absolute paths containing a username; repository names or URLs not in
PUBLIC_REPOS; any term in PROPRIETARY; and anything that reads as
customer, client, or internal-project content that a stranger should not
see.
Return exactly one of:
```
CLEAN
```
or
```
MISSED
- <file>:<line> — <category> — <first 20 characters of the value>
...
```
Do not paste more than 20 characters of any missed value. Do not comment
on the scrub's quality. Do not suggest fixes.
@@ -1,38 +0,0 @@
You are the scrubber. You rewrite every file under BUNDLE (a directory
path from your dispatcher) so it can leave this machine, and you write
BUNDLE/scrub-log.md. You never touch anything outside BUNDLE.
Inputs:
- BUNDLE: absolute path of the bundle directory.
- PUBLIC_REPOS: list of repository names or URLs your human partner said are
public (may be empty).
- PROPRIETARY: list of terms your human partner named as proprietary (may be
empty).
Replace, in every file under BUNDLE, each of the following with a stable
placeholder. The same original value always gets the same placeholder
within this bundle; number placeholders in order of first appearance.
| Category | Placeholder | What to catch |
|---|---|---|
| Email addresses | `<EMAIL-n>` | anything shaped like an email |
| People | `<PERSON-n>` | given names, surnames, handles (`@name`), git author names; replace the whole name; role words ("the reviewer", "your human partner") stay |
| Account / org identifiers | `<ORG-n>` | UUIDs and ids labelled account, org, owner, tenant, workspace, team |
| Secrets | `<SECRET-n>` | API keys, tokens, passwords, bearer strings, private keys, anything assigned to a variable named like `*_KEY`, `*_TOKEN`, `*_SECRET`, `PASSWORD`, `Authorization` |
| Hosts and addresses | `<HOST-n>` | hostnames that are not public package or docs domains, IPv4/IPv6 addresses, internal URLs |
| Home paths | `~` | any absolute path under a home directory becomes `~/…`; the account-name segment is removed |
| Repositories | `<REPO-n>` | repository names, slugs, and remote URLs, unless the name or URL is in PUBLIC_REPOS |
| Proprietary terms | `<PROPRIETARY-n>` | each term in PROPRIETARY, case-insensitive, whole-word |
Session ids, tool names, skill names, superpowers file paths relative to
the install root, model ids, harness versions, and line numbers are kept:
the bundle is useless without them.
Procedure:
1. `find BUNDLE -type f` and process every file, including
`environment.json` and `findings/*.md`.
2. Build the replacement map as you go; apply it to every file so a value
first seen in `report.md` is also replaced in `transcripts/`.
3. Write BUNDLE/scrub-log.md: a table of placeholder → category → number of
occurrences. Never write the original value into the log.
4. Return the scrub-log table and the list of files rewritten. Nothing else.
@@ -1,37 +0,0 @@
You are a matcher. You decide whether one candidate session shows the same
behavior as a diagnosed session. You do not modify any file.
Inputs:
- CASE: absolute path of the diagnosed session's case file. Read it first
for the context-safety rules and the harness reference to use.
- CANDIDATE: absolute path of one session transcript to examine.
- SIGNATURE: a list of markers. Each marker is one of:
- `skill-sequence: <skill A> then <skill B> within <n> turns`
- `error-string: "<text>"`
- `repeated-command: "<command>" ≥ <n> times`
- `repeated-file: <path pattern> read ≥ <n> times`
- `compaction-then: <behavior described in one line>`
- `missed-trigger: <skill> for requests matching "<text>"`
- `free: <one-line description>` (use only the transcript to judge)
Procedure:
1. `wc -lc` and the long-line check on CANDIDATE. Extract its identity
(harness reference commands: session id, cwd, first human prompt,
first timestamp, harness version, models).
2. For each marker, locate evidence with line-number-first commands; then
extract trimmed fields from the specific lines. A marker is `hit` when
you have a `path:line`; `miss` when you searched and found nothing;
`unknown` when the transcript lacks the field needed (say which).
3. Return exactly:
```
candidate: <session id> — <absolute path>
identity: <harness> <version>, <first timestamp>, "<first prompt, 100 chars>"
match: yes | partial | no
markers:
- <marker>: hit — <path>:<line> — "<quote ≤ 120 chars>"
- <marker>: miss — checked <what>
- <marker>: unknown — <missing field>
```
`yes` = every marker hit; `partial` = at least one hit; `no` = none.
@@ -1,69 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Skill timeline
Build the per-human-turn record of skill and plugin use, then look for gaps.
1. List the human prompts with line numbers and timestamps.
2. List every skill invocation (Claude Code: `Skill` tool_use `input.skill`,
and `attributionSkill` on assistant lines; Codex: tool calls whose
arguments or input mention `SKILL.md`; other harnesses: reads of files
named `SKILL.md`). Record the line, the skill name, and the human turn
it happened in.
3. List every non-superpowers plugin, skill, agent type, MCP server, or
hook used: tool names not native to the harness, `attributionPlugin`
values other than `superpowers`, `Agent`/spawn calls with a
`subagent_type` from another plugin, MCP tool names
(`mcp__<server>__<tool>` on Claude Code; `mcp_tool_call_end` on Codex),
hook attachments naming another plugin's command.
4. For each human turn, compare the request text against the trigger
descriptions of the superpowers skills installed (read
`<install root>/skills/*/SKILL.md` frontmatter `description` lines; the
install root is in the case file). Report as findings:
- a skill invoked, with the request that preceded it (one finding per
invocation is fine when there are few; group by skill when many);
- a turn whose request matches a skill's trigger description with no
invocation in that turn (state which description matched and quote
the request);
- a skill invoked one or more turns after the matching request (late);
- each non-superpowers plugin/skill/tool used, with where.
Do not say whether a missed or late trigger was wrong. Report the match
and the absence; the reader decides.
@@ -1,66 +0,0 @@
You are an analyst subagent. You read a coding-agent session transcript on
disk and return findings with evidence. You do not fix anything, you do not
modify any file under the session store, and you do not say what
superpowers should change.
Inputs (from your dispatcher):
- CASE: absolute path of the case file. Read it first. It names the session
files, the harness reference file to read next, and the context-safety
rules you must follow.
- RANGE (optional): a turn range or line range. If present, analyze only
that range and say so in your Checked line.
Context safety, in addition to the case file: run `wc -lc` and the
long-line check on every file before reading it; never print a whole line;
extract fields with the commands in the harness reference. If a command
returns more than 500 characters for one record, narrow it. "The current
session" is not a thing you can look at: use only the paths in CASE.
Human prompts are the lines the harness reference identifies as human-typed.
Hook output, system reminders, and tool results are not human prompts. In a
subagent transcript, "user" is the parent agent.
Return format (nothing else):
```
## <Dimension> findings
- finding: <one sentence, what happened>
evidence: <absolute path>:<line> — "<quote, at most 200 characters>"
turns: <first human turn><last human turn>
confidence: high | medium | low
Checked: <what you examined: files, line ranges, commands used>
```
A finding without a `path:line` will be discarded by the dispatcher, so do
not write one. If you found nothing, return `- none found` and the Checked
line.
Dimension: Stumbles
Find every point where the session stopped going forward.
Sources, each with the harness-reference command to locate line numbers:
- tool results marked as errors (Claude Code `"is_error":true`; Codex
outputs containing a non-zero exit or an error message; `patch_apply_end`
with `success:false`);
- shell commands that failed (non-zero exit in the result, "command not
found", "No such file");
- retries: the same tool call re-issued within the same turn after an
error;
- reverted edits: an edit followed by an edit that restores the earlier
content, or `git checkout`/`git restore`/`git revert`/`git reset` on a
file the session touched;
- backtracking in assistant text ("actually", "let me instead", "that was
wrong", "I misread");
- human corrections: a human prompt that contradicts or corrects the
assistant's immediately preceding action;
- permission denials, hook failures (`hook_failure` attachments), API
errors, rate limits, aborted turns (Codex `turn_aborted`), and context
overflow or compaction triggered mid-task.
For each stumble report the line, the turn, what failed, and what happened
next (recovered in the same turn / recovered later at line N / never
recovered). Group identical repeated failures into one finding with a
count.
@@ -1,105 +0,0 @@
# Claude Code session store
Verified against: Claude Code 2.1.247 (transcript `version` field), macOS.
When a field below is missing from the file in front of you, trust the file
and say so in coverage notes.
## Where
- Main transcript: `~/.claude/projects/<cwd-slug>/<sessionId>.jsonl`, where
`<cwd-slug>` is the working directory with every `/` replaced by `-`
(e.g. `/tmp/work``-tmp-work`).
- Subagent transcripts: `~/.claude/projects/<cwd-slug>/<sessionId>/subagents/agent-<agentId>.jsonl`,
each with a sibling `agent-<agentId>.meta.json`
(`agentType`, `description`, `toolUseId`, `spawnDepth`, optional `model`).
- Plugin registry: `~/.claude/plugins/installed_plugins.json` — per plugin:
`installPath`, `version`, `installedAt`, `lastUpdated`, `gitCommitSha`.
- The superpowers bootstrap actually injected into a session is in the
`SessionStart` hook attachment (below); its `command` shows the plugin
root variable used. A dev checkout loaded with `--plugin-dir` will not be
in the registry, so report both the registry entry and the hook evidence.
## Which file is the current session
The most recently modified `.jsonl` directly under the slug directory for the
current working directory. Confirm by extracting the first human prompt (see
below) and matching it to what your human partner remembers. If two files
are close in mtime, show both first prompts and ask.
## Line types
Every line is one JSON object. `type` values seen: `user`, `assistant`,
`attachment`, `system`, plus session-level records (`permission-mode`,
`mode`, `bridge-session`, `last-prompt`, `ai-title`, `atis-latch`,
`pr-link`, `queue-operation`, `relocated`, `worktree-state`).
Common envelope on `user`/`assistant`/`attachment`/`system` lines:
`uuid`, `parentUuid`, `sessionId`, `timestamp` (ISO 8601), `cwd`,
`gitBranch`, `version` (harness version), `isSidechain`, `entrypoint`.
| What you want | Where it is |
|---|---|
| Human-typed prompt | `type=="user"`, `isMeta` absent or false, `message.content` is a string or a list whose first block is `type:"text"`. Lines whose first block is `tool_result` are tool results, not prompts. `<system-reminder>` text inside a prompt is injected, not typed. Text beginning with `<task-notification>`, `<command-name>`, `<local-command-stdout>`, `<system-reminder>`, or `This session is being continued from a previous conversation` is harness-injected too, even though `isMeta` is absent on those lines — exclude them or your human-turn count will be several times too high. |
| Human-typed prompt queued mid-turn | `type=="attachment"`, `attachment.type=="queued_command"`, `attachment.origin.kind=="human"`, text in `attachment.prompt`. These are typed while a turn is running and never appear as standalone `user` lines, so they are missing from the list above. Add them to the timeline. |
| Assistant text / tool calls | `type=="assistant"`, `message.content[]` blocks of `type:"text"` or `type:"tool_use"` (`id`, `name`, `input`). |
| Tool result | `type=="user"`, `message.content[0].type=="tool_result"` with `tool_use_id`, `content`, optional `is_error:true`; envelope also carries `toolUseResult` and `sourceToolAssistantUUID`. |
| Model | `message.model` on assistant lines. |
| Tokens | `message.usage` on assistant lines: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`. |
| Skill invocation | `tool_use` block with `name:"Skill"` and `input.skill` (e.g. `superpowers:brainstorming`); the tool result line has `toolUseResult.commandName`. |
| Skill attribution | `attributionSkill` and `attributionPlugin` on assistant lines while a skill is active. |
| Subagent dispatch | `tool_use` with `name:"Agent"` (`input.description`, `input.subagent_type`, `input.prompt`); the subagent's own file is matched by `toolUseId` in its `.meta.json`. Subagent lines have `isSidechain:true` and `agentId`. |
| Hook output | `type=="attachment"`, `attachment.type` `hook_success`/`hook_failure`, `attachment.hookName` (e.g. `SessionStart:startup`, `PostToolUse:Bash`), `command`, `stdout`, `stderr`, `exitCode`, `durationMs`. |
| Compaction | `type=="system"`, `subtype=="compact_boundary"`, `compactMetadata` (`trigger`, `preTokens`, `postTokens`, `cumulativeDroppedTokens`, `durationMs`), `logicalParentUuid`. |
| Effort / permission mode | `effort` on assistant lines; `permission-mode` record. |
## Safe extraction
Lines can exceed a megabyte. Never print a whole line. Check size first:
```bash
F=~/.claude/projects/<slug>/<id>.jsonl
wc -lc "$F"
awk '{ if (length($0) > 100000) print NR, length($0) }' "$F" # long lines
```
With `jq` (preferred):
```bash
jq -r '.type' "$F" | sort | uniq -c # line-type census
jq -r 'select(.type=="user" and .isMeta!=true and ((.message.content|type)=="string" or .message.content[0].type=="text"))
| select((.message.content|if type=="string" then . else (.[0].text // "") end)
| test("^(<task-notification>|<command-name>|<local-command-stdout>|<system-reminder>|This session is being continued)") | not)
| "\(input_line_number)\t\(.timestamp)\t\((.message.content|if type=="string" then . else .[0].text end)[0:160])"' "$F" # human prompts
jq -r 'select(.type=="attachment" and .attachment.type=="queued_command" and .attachment.origin.kind=="human")
| "\(input_line_number)\t\(.timestamp)\t\(.attachment.prompt[0:160])"' "$F" # human prompts queued mid-turn; merge with the list above
jq -c 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use")
| {name, id, input: (.input|tostring|.[0:120])}' "$F" # tool calls
jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use" and .name=="Skill") | .input.skill' "$F" # skill invocations
jq -c 'select(.type=="assistant") | {ts:.timestamp, model:.message.model, skill:.attributionSkill,
u:(.message.usage|{input_tokens,output_tokens,cache_read_input_tokens,cache_creation_input_tokens})}' "$F" # per-message usage
jq -c 'select(.subtype=="compact_boundary") | {line:input_line_number, ts:.timestamp,
m:(.compactMetadata|{trigger,preTokens,postTokens,cumulativeDroppedTokens,durationMs})}' "$F" # compactions (full compactMetadata also has UUID lists; keep this trimmed)
jq -c 'select(.type=="attachment" and (.attachment.type|startswith("hook"))) | {line:input_line_number, hook:.attachment.hookName, exit:.attachment.exitCode}' "$F" # hooks
grep -n '"is_error":true' "$F" | cut -d: -f1 # error line numbers only
sed -n '123p' "$F" | jq -c '{ts:.timestamp, first:((.message.content // "") as $c
| ($c | if type=="array" then ($c[0] // "") else $c end) | tostring | .[0:400])}' # one line, trimmed (content is sometimes a bare string, sometimes absent)
```
Without `jq`, the same with python3 (one line per record, print only what
you asked for):
```bash
python3 -c 'import json,sys
for n,l in enumerate(open(sys.argv[1]),1):
o=json.loads(l)
if o.get("type")=="assistant":
for b in o["message"].get("content",[]):
if b.get("type")=="tool_use": print(n, b["name"], str(b.get("input"))[:120])' "$F"
```
## Subagents
List `~/.claude/projects/<slug>/<id>/subagents/`. For each `agent-*.meta.json`
print `agentType`, `description`, `model`; the matching `.jsonl` is that
subagent's transcript and follows the same line format. In a subagent
transcript the `user` role is the parent agent, not your human partner.
@@ -1,82 +0,0 @@
# Codex session store
Verified against: Codex CLI 0.146.0, 0.147.0 and 0.149.0-alpha.4.1 rollouts
(`cli_version` in `session_meta`), macOS. When a field below is missing from
the file in front of you, trust the file and say so in coverage notes.
## Where
`~/.codex/sessions/YYYY/MM/DD/rollout-<ISO-timestamp>-<thread-id>.jsonl`.
Subagent threads are separate rollout files whose `session_meta.payload`
has `thread_source: "subagent"` and `source.subagent.thread_spawn.parent_thread_id`
pointing at the parent thread id. Root sessions have `thread_source: "user"`.
## Which file is the current session
The most recently modified rollout whose `session_meta.payload.cwd` is the
current working directory and whose `thread_source` is `user`. Confirm by
matching the first `user_message` event to what your human partner
remembers. Newer rollouts may carry no `user_message` event at all: when
that command returns nothing, fall back to `response_item` messages with
`role:"user"` (see Human-typed prompt below) and confirm against the first
of those instead.
## Line types
Every line is `{timestamp, type, payload}` (some also carry `ordinal`).
`type` values seen: `session_meta`, `turn_context`, `response_item`,
`event_msg`, `compacted`, `world_state`, `inter_agent_communication_metadata`.
| What you want | Where it is |
|---|---|
| Session identity | `session_meta.payload`: `id`, `session_id`, `cwd`, `originator` (e.g. `Codex Desktop`), `cli_version`, `model_provider`, `thread_source`, `source`, `git` (`commit_hash`, `branch`, `repository_url`), `base_instructions.text`. |
| Model per turn | `turn_context.payload`: `turn_id`, `model`, `effort`, `cwd`, `approval_policy`, `sandbox_policy`, `multi_agent_version`. Also `event_msg` `thread_settings_applied`. |
| Human-typed prompt | `event_msg` with `payload.type=="user_message"`: `payload.message`. When that returns nothing — seen on `thread_source: "user"` Codex Desktop rollouts at `cli_version 0.149.0-alpha.4.1`, and on subagent rollouts — fall back to `response_item` messages with `payload.role=="user"`, text in `payload.content[0].text`. `role:"developer"` messages are injected boilerplate, not typed, and so is any fallback text that begins with a tag such as `<subagent_notification>`, `<environment_context>`, `<skill>` or `<recommended_plugins>`. On a subagent rollout the fallback text is the parent agent's dispatch prompt, not your human partner's. |
| Assistant text | `event_msg` `agent_message` (`payload.message`, `payload.phase`) or `response_item` `message` with `role:"assistant"`. |
| Tool calls | `response_item` with `payload.type` `function_call` (`name`, `arguments`, `call_id`) or `custom_tool_call` (`name`, `input`, `call_id`); outputs are `function_call_output` / `custom_tool_call_output` matched by `call_id`. Also `event_msg` `patch_apply_end` (`success`, `changes`), `web_search_end`, `mcp_tool_call_end` (`invocation.server`, `invocation.tool`). |
| Turn timing | `event_msg` `task_started` (`turn_id`, `started_at`, `model_context_window`) and `task_complete` (`duration_ms`, `time_to_first_token_ms`, `last_agent_message`); `turn_aborted` (`reason`, `duration_ms`). |
| Tokens | `event_msg` `token_count`: `payload.info.total_token_usage` (cumulative; keys include `input_tokens`, `cached_input_tokens`, `output_tokens`) and `payload.rate_limits`. |
| Compaction | a `compacted` line (`window_id`, `previous_window_id`, `replacement_history`) and an `event_msg` `context_compacted`. |
| Subagents | `event_msg` `sub_agent_activity` (`agent_thread_id`, `agent_path`, `kind`); `response_item` `agent_message` with `author`/`recipient`; the child's own rollout file (see Where). |
| Skill use | No attribution field. Look for `SKILL.md` in `function_call.arguments` / `custom_tool_call.input` and in `world_state`/`session_meta` instruction text. |
| Reasoning | `response_item` `reasoning` (`summary[].text`; `encrypted_content` is opaque). |
## Safe extraction
Rollouts reach hundreds of megabytes; `compacted` lines embed whole
histories. Never print a whole line. Check size first:
```bash
F=~/.codex/sessions/YYYY/MM/DD/rollout-....jsonl
wc -lc "$F"
awk '{ if (length($0) > 100000) print NR, length($0) }' "$F"
```
With `jq`:
```bash
head -1 "$F" | jq '.payload | {id, cwd, originator, cli_version, model_provider, thread_source, git}' # identity
jq -r '.type + "/" + (.payload.type // "")' "$F" | sort | uniq -c # census
jq -r 'select(.type=="event_msg" and .payload.type=="user_message") | "\(input_line_number)\t\(.timestamp)\t\(.payload.message[0:160])"' "$F" # human prompts
jq -r 'select(.type=="response_item" and .payload.type=="message" and .payload.role=="user")
| "\(input_line_number)\t\(.timestamp)\t\((.payload.content[0].text // "")[0:160])"' "$F" # human prompts, fallback when the line above returns nothing; skip rows whose text starts with a `<tag>`
jq -r 'select(.type=="turn_context") | "\(.timestamp)\t\(.payload.model)\t\(.payload.effort)"' "$F" # model per turn
jq -c 'select(.type=="response_item" and (.payload.type=="function_call" or .payload.type=="custom_tool_call"))
| {line:input_line_number, name:.payload.name, args:((.payload.arguments // .payload.input)|tostring|.[0:120])}' "$F" # tool calls
jq -c 'select(.payload.type=="task_complete" or .payload.type=="turn_aborted") | {ts:.timestamp, type:.payload.type, ms:.payload.duration_ms}' "$F" # turn timing
jq -c 'select(.payload.type=="token_count") | {ts:.timestamp, t:.payload.info.total_token_usage}' "$F" # tokens (cumulative)
grep -n '"type":"compacted"\|"context_compacted"' "$F" | cut -d: -f1 # compaction line numbers
grep -n 'SKILL\.md' "$F" | cut -d: -f1 # skill-read line numbers
sed -n '123p' "$F" | jq -c '{ts:.timestamp, type, p:(.payload|tostring|.[0:400])}' # one line, trimmed
```
Find a thread's subagent rollouts (filenames only, never content):
```bash
grep -l '"parent_thread_id":"<thread-id>"' ~/.codex/sessions/*/*/*/rollout-*.jsonl
```
A subagent rollout can carry no `event_msg` `user_message` at all — the
parent agent's dispatch prompt instead shows up as a `response_item`
`message` with `role:"user"`. If a `user_message` event is present, it is
from the parent agent, not your human partner.
@@ -1,30 +0,0 @@
# Other harnesses: discover, then report what you found
This file is for any harness without a verified reference in this
directory. You know your own harness better than this file does. Use that
knowledge, and write down exactly what you found so the report reader can
judge it.
## Procedure
1. **Ask the harness.** Many harnesses expose a session or history command
(`<harness> session list`, `/sessions`, a "resume" picker). Use it to get
the session id and, if shown, the file path.
2. **Look under the harness's config directory** (`~/.<harness>/`,
`~/.config/<harness>/`, `~/.local/share/<harness>/`) for `sessions`,
`history`, `chats`, `threads`, or `projects` directories holding `.jsonl`
or `.json` files.
3. **Confirm a candidate** by extracting its first human message with a
size-safe command (`head -c 2000`, or `jq` on the first record) and
matching it to what your human partner remembers. Never print whole
lines; treat every candidate like the verified stores: `wc -lc` and a
long-line check before anything else.
4. **Map the fields you need** by reading a handful of records with `jq -c
'keys'` or `head -c`: human prompt, assistant text, tool call and result,
model, harness version, timestamps, subagent linkage, compaction.
5. **Record in the case file and the report's coverage notes**: the store
path, the layout you inferred, which of the fields above you could and
could not find, and your confidence. Field-level claims in the report
are marked "inferred from the file, not a documented format".
6. **If you cannot find the store**, say so and ask your human partner for
the path. Do not guess a layout from another harness.
@@ -1,44 +0,0 @@
# Superpowers session diagnosis bundle
Session: <session-id>
Harness: <name> <version> Superpowers: <version> (<sha or "not a checkout">)
Redaction level: skeleton | evidence | full
Built: <ISO timestamp>
## What this is
A scrubbed record of a coding-agent session in which superpowers was
installed and something went wrong, prepared so that an agent or person
who was not present can decide whether superpowers contributed and, if so,
what to change. The report inside states what happened with `path:line`
evidence. By design it contains no diagnosis of superpowers and no proposed
fix; that is the reader's job.
## Files
- `report.md` — the diagnosis report (problem statement, verdict,
environment, sessions, timeline, findings, involvement, coverage notes).
- `case.md` — the case file the analysts worked from.
- `environment.json` — machine-readable copy of the environment section.
- `timeline.md` — the per-turn timeline.
- `findings/<dimension>.md` — raw analyst findings per dimension.
- `transcripts/<session-id>.md` — condensed per-turn rendering of each
examined session (never the raw JSONL). At *skeleton* level tool-result
bodies are replaced by `[tool result: <tool>, <bytes> bytes, exit <code>]`;
at *evidence* level bodies are kept only for events cited in findings; at
*full* level all bodies are kept.
- `scrub-log.md` — every placeholder used and its category (never the
original value).
## How to read it
Start with `report.md` §12, then §7 (involvement) and the evidence lines
it cites, then the matching turns in `transcripts/`. `path:line` references
point at the original files on the reporter's machine; the same line
numbers are preserved in the condensed transcripts as `[L<n>]` markers.
## Redaction
Placeholders look like `<EMAIL-1>`, `<PERSON-2>`, `<SECRET-3>`, `<HOST-4>`,
`<REPO-5>`, `<ORG-6>`, `<PROPRIETARY-7>`; home paths are rewritten to `~/…`. The same placeholder
always refers to the same original value within this bundle.
@@ -1,50 +0,0 @@
# Case: <session-id>
Workspace: ~/.superpowers/diagnosing-superpowers/<session-id>/
Created: <ISO timestamp>
## Problem statement (agreed with your human partner)
<One paragraph. Names the session(s), the turn range if known, what was
expected, what happened, and the observable that matters: wall-clock,
tokens, repeated actions, a specific unexpected action.>
Goal is a superpowers bug report: yes | no
## Sessions
| Role | Session id | Absolute path | Lines | Bytes | Longest line (bytes) | First prompt (first 120 chars) | First timestamp |
|---|---|---|---|---|---|---|---|
| main | | | | | | | |
| subagent | | | | | | | |
Rejected candidates: <id — path — why rejected>, or "none".
Session still running at read time: yes | no (mtime <ISO>, lines <N>)
## Environment
- OS: <name and version>
- Harness: <name> <version>
- Models seen: <model id — where (main / subagent id)>
- Superpowers install root: <path>; version <x.y.z>; git sha <sha or "not a checkout">
- Skill files read or injected during the session:
| File (relative to install root) | sha1 (current file) | mtime newer than session? |
|---|---|---|
- Other plugins / extensions / MCP servers configured: <list, or "none found">
- Instruction files present (paths only): <list>
## Context-safety rules for every reader of these files
- Check `wc -lc` and long lines (`awk '{ if (length($0) > 100000) print NR, length($0) }'`) before reading.
- Never `cat` or `grep` for content. Line numbers and counts first
(`grep -n … | cut -d: -f1`), then small fields from specific lines
(`sed -n Np | jq -c '{…}'` or `| cut -c1-500`).
- Read-only: never modify, move, or delete a session file.
- In a subagent transcript, "user" is the parent agent.
## Harness reference to use
<references/claude-code-sessions.md | references/codex-sessions.md | references/other-harnesses.md>
@@ -1,49 +0,0 @@
- [x] I searched existing issues and this is not a duplicate (searched: <query terms>; closest: <#n title, or "none">)
## Environment (required)
| Field | Value |
|-------|-------|
| Superpowers version | <version> (<sha or "not a checkout">) |
| Harness (Claude Code, Cursor, etc.) | <harness> |
| Harness version | <version> |
| Your model + version | <model ids seen> |
| All plugins installed | <list> |
| OS + shell | <os version>, <shell> |
## Is this a Superpowers issue or a platform issue?
- [ ] I confirmed this issue does not occur without Superpowers installed
Not reproduced without superpowers. Evidence for involvement is below;
the reporter has not established cause.
## What happened?
<Problem statement, then the triage verdict, with `path:line` citations
rewritten as `transcript line <n>`.>
## Steps to reproduce
1. <first human prompt, scrubbed>
2. <the turns leading to the problem, one line each>
3. <the observable>
## Expected behavior
<from the problem statement>
## Actual behavior
<from the triage verdict>
## Debug log or conversation transcript
Session id(s): <ids>. A scrubbed bundle (redaction level: <level>) is
attached to this issue by the reporter, or available on request.
Superpowers involvement per the diagnosis report: <possible | likely>, with
evidence at <transcript lines>. This report does not propose a fix.
---
Filed with the `diagnosing-superpowers` skill. Model, harness, harness
version, and installed plugins are listed above.
@@ -1,78 +0,0 @@
# Session diagnosis: <session-id>
Report path: ~/.superpowers/diagnosing-superpowers/<session-id>/report.md
Written: <ISO timestamp>
## 1. Problem statement (REQUIRED)
<Copied from the case file.>
## 2. Triage verdict (REQUIRED)
<What the evidence shows happened around the reported problem. Prose, with
`path:line` after every claim. State confidence: high / medium / low, and
what would raise it. No statement about what superpowers should do.>
## 3. Environment (REQUIRED)
- OS:
- Harness and version:
- Models seen:
- Superpowers install root / version / git sha:
- Skill files read or injected (sha1 table from the case file):
- Other plugins, extensions, MCP servers:
- Instruction files present (paths only):
## 4. Sessions examined (REQUIRED)
| Role | Session id | Absolute path | Lines | Bytes |
|---|---|---|---|---|
Rejected candidates: <id — path — why>, or "none".
## 5. Timeline (REQUIRED)
One row per human-typed prompt. Events column lists skills invoked,
subagents dispatched, compaction, errors, resumes, aborts.
| Turn | Line | Time | Request (one line) | Events |
|---|---|---|---|---|
## 6. Findings (REQUIRED, one subsection per dimension)
Each finding:
```
- finding: <one sentence>
evidence: <path:line> — "<short quote>"
turns: <first><last>
confidence: high | medium | low
```
A dimension with nothing to report says `none found — checked: <what was checked>`.
### 6.1 Skill timeline
### 6.2 Plan adherence
### 6.3 Repeated work
### 6.4 Stumbles
### 6.5 Quality evidence
### 6.6 Request conflicts
### 6.7 Cost and time
### 6.8 Other plugins and skills used
## 7. Superpowers involvement (REQUIRED)
not indicated | possible | likely
Evidence lines: <path:line list>. This section states involvement only. It
does not name a defect and does not propose a change.
## 8. Coverage notes (REQUIRED)
- Not read: <ranges, files, and why>
- Harness features unavailable: <list or none>
- Session was in progress at read time: yes/no
- For your human partner to double-check: <list or none>
## 9. Similar sessions (only when requested)
| Session id | Path | Date | Harness | Matched | Did not match |
|---|---|---|---|---|---|
+53 -2
View File
@@ -173,6 +173,14 @@ its own text agrees with itself — the tests it specifies against the code it
specifies, the files it creates against the files it later touches. "The scan
is clean" without those rows is not a scan you ran.
**When the plan's header declares `Plan shape: skeleton-first`,** the
table gets a final section: the DISPATCH PLAN — group the pending tasks
into waves. Tasks in the same wave are mutually file-disjoint and consume
no interface still under construction — dispatch each wave's implementers
concurrently, one worktree per task, and integrate before the next wave;
tasks that fail those conditions serialize. On a skeleton-first plan, a
scan without a dispatch plan is not a scan you ran.
Write the table to the ledger. Rule on everything you find before execution
begins — each finding against the plan text that mandates it — and record
each ruling in the ledger. If the scan is clean, proceed without comment.
@@ -187,6 +195,10 @@ Use the least powerful model that can handle each role to conserve cost and incr
**Mechanical implementation tasks** (isolated functions, clear specs, 1-2 files): use a fast, cheap model. Most implementation tasks are mechanical when the plan is well-specified.
When a task carries a **Tier:** field, follow it — the planner already
ruled: mechanical → the cheapest available model; judgment → a standard
model. Do not re-litigate the tier at dispatch.
**Integration and judgment tasks** (multi-file coordination, pattern matching, debugging): use a standard model.
**Architecture and design tasks**: use the most capable available model.
@@ -208,7 +220,10 @@ most expensive — which silently defeats this section.
**Turn count beats token price.** Wall-clock and context cost scale with how
many turns a subagent takes, and the cheapest models routinely take 2-3× the
turns on multi-step work — costing more overall. Use a mid-tier model as the
floor for reviewers and for implementers working from prose descriptions.
floor for reviewers and for implementers working from task contracts or
prose descriptions — unless the task's Tier line says mechanical: the
planner has already ruled the deliverable fully specified, so treat a
mechanical-tier contract like spelled-out content.
When the task's plan text contains the complete code to write, the
implementation is transcription plus testing: use the cheapest tier for
that implementer. Single-file mechanical fixes also take the cheapest tier.
@@ -259,7 +274,13 @@ and fix-round diffs need it.
know; (4) your resolution of any ambiguity you noticed in the brief;
(5) the report-file path and report contract. Exact values (numbers,
magic strings, signatures, test cases) appear only in the brief. Never
make a subagent read the whole plan file.
make a subagent read the whole plan file. When the brief is a contract
(goal, success criteria, interfaces) rather than written-out code, item
(3) also carries the elaboration the contract leaves to dispatch time:
the interfaces as actually built by completed tasks, environment facts
and discoveries from earlier reports, and any amendment rulings. There
the success criteria name the cases the tests must cover, and the
implementer designs its own code and tests within the contract.
- **Report file:** name the implementer's report file after the brief
(brief `…/task-N-brief.md` → report `…/task-N-report.md`) and put it in
the dispatch prompt. The implementer writes the full report there and
@@ -280,6 +301,26 @@ and fix-round diffs need it.
- Record the implementer's agent identity from the dispatch result —
fix-loop rounds 1-3 resume this agent.
- Never dispatch multiple implementation subagents in parallel (conflicts).
The one exception is a skeleton-first plan whose dispatch plan shows two
or more pending tasks mutually file-disjoint with none consuming an
interface still under construction. Dispatch those implementers
concurrently, each in its own worktree:
- Record the integration base commit in the ledger before the first
concurrent dispatch.
- Create one worktree per concurrent task off that base
(`git worktree add <repo-root>/.worktrees/task-<N> -b task-<N>
<base>`); each dispatch's `Work from:` is its own worktree, and its
BASE is that worktree's HEAD.
- Review each task's diff as usual when it reports. Integrate reviewed
branches in plan order: merge each into the integration branch
(`git merge --no-ff task-<N>`), and run that task's verification
commands after each merge.
- A merge conflict or post-merge verification failure is that task's
fix-loop round 1: rebase the task branch onto the current
integration head in its worktree, then resume its implementer
there. Never resolve conflicts yourself.
- Remove each worktree (`git worktree remove`) once its branch is
integrated, and record the integrated range in the ledger as usual.
Template: [implementer-prompt.md](implementer-prompt.md)
@@ -438,6 +479,16 @@ message as your other bookkeeping:
- `Task <N>: complete (commits <base7>..<head7>, <K> parked)` after a
tripped breaker
**On a skeleton-first plan,** write one plan-check line with the
completion line. Re-read the remaining tasks against what this task
actually established — interfaces as built, environment facts,
discoveries in the report — and append either `Plan holds` or
`Amendment: Task <M>: <what changes and why>` to the ledger. An
amendment is plan authority applied at the plan layer: from then on the
amended text IS the plan's text, and it rides into every affected task's
dispatch under item (3). Never dispatch a task whose brief a completed
task's report has already invalidated.
Then mark the todo complete and move on. Never move to the next task while
the review has open Critical/Important issues that are neither fixed nor
parked-with-ruling at the cap.
@@ -5,8 +5,11 @@ Use this template when dispatching an implementer subagent.
```
Subagent (general-purpose):
description: "Implement Task N: [task name]"
model: [MODEL — REQUIRED: choose per SKILL.md Model Selection; an omitted
model silently inherits the session's most expensive one]
model: [MODEL — REQUIRED: when the brief carries a Tier line, set from it:
mechanical → the cheapest model the subagent tool offers; judgment →
a standard mid-tier model. Otherwise choose per SKILL.md Model
Selection. An omitted model silently inherits the session's most
expensive one]
prompt: |
You are implementing Task N: [task name]
+24
View File
@@ -22,6 +22,30 @@ Assume they are a skilled developer, but know almost nothing about our toolset o
If the spec covers multiple independent subsystems, it should have been broken into sub-project specs during brainstorming. If it wasn't, suggest breaking this into separate plans — one per subsystem. Each plan should produce working, testable software on its own.
## Two Plan Shapes
Before mapping files, classify the plan's shape and say the
classification out loud — "this composes three subsystems, so I'll plan
it skeleton-first" — so your human partner can override it:
- **Task-by-task (default)** — tasks build the feature a component at a
time, each step carrying the actual content the engineer needs. Use it
for changes to code that already exists, for a spec that touches one
subsystem, and whenever the alternative's conditions do not clearly
hold. The rest of this skill describes this shape.
- **Skeleton-first (alternative)** — Task 1 is the thinnest end-to-end
slice through every subsystem the spec composes; later tasks widen it
one component at a time, each from a contract rather than written-out
code. Use it when the spec composes more than one subsystem AND a
running end-to-end slice early is worth a longer total build. Read
[skeleton-first-plans.md](skeleton-first-plans.md) before writing one
— it adds one line to the plan header and replaces this skill's task
granularity, task template, and plan-failure list.
When in doubt, plan task-by-task. Skeleton-first buys an earlier running
system and pays for it in total wall clock; it is a trade, not an
upgrade.
## File Structure
Before defining tasks, map out which files will be created or modified and what each one is responsible for. This is where decomposition decisions get locked in.
@@ -0,0 +1,131 @@
# Skeleton-First Plans
The alternative plan shape from writing-plans' Two Plan Shapes router.
Each section below replaces the same-named section of
[SKILL.md](SKILL.md); everything SKILL.md says that is not named here
still binds — Scope Check, File Structure, Task Right-Sizing, the plan
header, Self-Review, and the Execution Handoff.
## Overview
Write a plan that carries the decisions, not the keystrokes:
decomposition, file structure, interfaces, constraints, and a precise
contract per task. Assume the engineer is skilled and designs their own
code and tests from a precise contract, but knows nothing about our
codebase, toolset, or problem domain — every name, path, constraint, and
behavior they must match is stated explicitly. DRY. YAGNI. TDD.
Frequent commits.
## When This Shape Fits
Use it when the spec composes more than one subsystem and a running
end-to-end slice early is worth a longer total build: the value arrives
as soon as real input reaches real output, and every later task widens
something that already runs.
Do not use it for a change to one subsystem, or when the whole point is
to land the finished thing as fast as possible. This shape spends its
first task on a slice that does almost nothing, and it spends planning
effort on contracts and interfaces the task-by-task shape gets for free
by writing the code out.
## Plan Document Header
The header is SKILL.md's, plus one line directly under the **Goal:**
line, which is how executors know which shape they are running:
```markdown
**Plan shape:** skeleton-first
```
## Walking Skeleton First
Task 1 builds the thinnest end-to-end slice through every subsystem the
spec composes — real input to real output — before any task deepens a
single layer; later tasks widen the skeleton.
The test of a skeleton is that it runs. A first task that builds the
data loader, the schema, or the config layer is a foundation, not a
skeleton: nothing runs until something above it exists. A skeleton
reaches the output — thinly, with one real case — through every
subsystem the spec names.
## Task Contracts, Not Task Scripts
A task states WHAT must exist when it is done, precisely enough that a
skilled engineer can build it without asking you anything, without
prescribing HOW:
- **Goal:** one short paragraph naming the deliverable and its role in
the feature.
- **Success criteria:** concrete, checkable behaviors — exact commands
to run and what they must show, the cases tests must cover (including
failure cases), constraints that bind the implementation.
- **Notes:** what the engineer needs and cannot discover alone — spec
sections to read, files worth reading first, known pitfalls.
The Interfaces block carries the exact names, signatures, and types;
the success criteria carry the behaviors; the engineer supplies the
code and the test design. TDD and frequent commits remain required.
## Task Structure
````markdown
### Task N: [Component Name]
**Files:**
- Create: `exact/path/to/file.py`
- Modify: `exact/path/to/existing.py:123-145`
- Test: `tests/exact/path/to/test.py`
**Interfaces:**
- Consumes: [what this task uses from earlier tasks — exact signatures]
- Produces: [what later tasks rely on — exact function names, parameter
and return types. A task's implementer sees only their own task; this
block is how they learn the names and types neighboring tasks use.]
**Goal:** [one paragraph — the deliverable and its role in the feature]
**Success criteria:**
- Run: `pytest tests/exact/path/to/test.py -v` — all tests pass; tests
cover [the specific behaviors and failure cases, named concretely]
- [observable behavior the deliverable must exhibit, with the exact
command or input/output that demonstrates it]
- [constraint that binds the implementation, copied from the spec]
**Notes:** [spec sections to read; files to read first; known pitfalls]
**Tier:** mechanical | judgment. Mechanical = the deliverable is fully
specified by Files + Interfaces + success criteria above (most tasks in
a well-specified plan are mechanical); judgment = multi-file
coordination, debugging, or real design latitude remains. The
implementer's model follows this field — mark it deliberately.
**Commit:** one commit ending the task; message named here.
````
## No Vague Contracts
Every contract must be checkable by someone who did not write it. These
are **plan failures** — never write them:
- "TBD", "TODO", "implement later", "fill in details"
- Goals naming activity instead of a deliverable ("improve error handling")
- Success criteria with no observable check ("works correctly", "handles edge cases")
- Interfaces blocks omitting a name, signature, or type another task consumes
- "Similar to Task N" (state this task's own contract in full — the engineer may be reading tasks out of order)
- References to types, functions, or methods not defined in any task's Interfaces block
## Self-Review
Run SKILL.md's Self-Review checklist, reading step 2 against "No Vague
Contracts" above rather than "No Placeholders".
## Red Flags
| Thought | Reality |
|---------|---------|
| "Task 1 is the data loader — that's the foundation" | A foundation is a layer. The skeleton runs real input to real output through every subsystem the spec names, thinly. |
| "The skeleton can return a hardcoded value for now" | It may be thin, but the path must be real: real input, real wiring, real output. A hardcoded response tests nothing end to end. |
| "A contract without the code is vague" | Vague is an uncheckable success criterion. Exact names, exact commands, exact expected output — no code. |
| "I'll write the test code into the task to be safe" | The success criteria name the cases; the implementer designs the tests. Written-out tests are the task-by-task shape. |
| "Skeleton-first is the better shape, so I'll use it here" | It costs total wall clock. Without more than one subsystem and a reason to want an early running slice, plan task-by-task. |
@@ -1,128 +0,0 @@
#!/usr/bin/env bash
# Structural checks for skills/diagnosing-superpowers. Behavior is tested by
# the scenarios in CREATION-LOG.md; this script only checks the things a
# shell can check: frontmatter, referenced files exist, no local paths or
# names leaked into shipped files, SKILL.md word budget.
set -u
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
SKILL_DIR="$REPO_ROOT/skills/diagnosing-superpowers"
SKILL_MD="$SKILL_DIR/SKILL.md"
WORD_BUDGET=900
PASSES=0
FAILURES=0
pass() { echo " [PASS] $1"; PASSES=$((PASSES + 1)); }
fail() { echo " [FAIL] $1"; FAILURES=$((FAILURES + 1)); }
echo "diagnosing-superpowers structure"
# --- SKILL.md frontmatter -------------------------------------------------
if [ -f "$SKILL_MD" ]; then
pass "SKILL.md exists"
frontmatter="$(awk 'NR==1 && $0!="---"{exit} NR>1 && $0=="---"{exit} NR>1{print}' "$SKILL_MD")"
if printf '%s\n' "$frontmatter" | grep -q '^name: diagnosing-superpowers$'; then
pass "frontmatter name is diagnosing-superpowers"
else
fail "frontmatter name is diagnosing-superpowers"
fi
description="$(printf '%s\n' "$frontmatter" | awk '/^description:/{sub(/^description:[ ]*/,""); print; found=1; next} found && /^[ ]/{print} found && !/^[ ]/{exit}' | tr '\n' ' ')"
if printf '%s' "$description" | grep -q '^Use when'; then
pass "description starts with 'Use when'"
else
fail "description starts with 'Use when' (got: ${description:0:60})"
fi
if [ "${#description}" -le 1024 ]; then
pass "description under 1024 characters"
else
fail "description under 1024 characters (${#description})"
fi
for banned in "dispatch" "then" "step"; do
if printf '%s' "$description" | grep -qiw "$banned"; then
fail "description contains workflow word '$banned'"
else
pass "description avoids workflow word '$banned'"
fi
done
# --- word budget --------------------------------------------------------
body_words="$(awk 'BEGIN{fm=0} NR==1 && $0=="---"{fm=1; next} fm==1 && $0=="---"{fm=2; next} fm==2{print}' "$SKILL_MD" | wc -w | tr -d ' ')"
if [ "$body_words" -le "$WORD_BUDGET" ]; then
pass "SKILL.md body within $WORD_BUDGET words ($body_words)"
else
fail "SKILL.md body within $WORD_BUDGET words ($body_words)"
fi
# --- required sections --------------------------------------------------
for heading in "## Hard rules" "## Red Flags"; do
if grep -q "^$heading" "$SKILL_MD"; then
pass "SKILL.md has section '$heading'"
else
fail "SKILL.md has section '$heading'"
fi
done
# --- every referenced skill file exists --------------------------------
while IFS= read -r ref; do
if [ -f "$SKILL_DIR/$ref" ]; then
pass "referenced file exists: $ref"
else
fail "referenced file exists: $ref"
fi
done < <(grep -o '\(references\|prompts\|templates\)/[A-Za-z0-9._-]*\.md' "$SKILL_MD" | sort -u)
else
fail "SKILL.md exists"
fi
# --- expected files -------------------------------------------------------
expected_files=(
references/claude-code-sessions.md
references/codex-sessions.md
references/other-harnesses.md
prompts/skill-timeline.md
prompts/plan-adherence.md
prompts/repeated-work.md
prompts/stumbles.md
prompts/quality-evidence.md
prompts/request-conflicts.md
prompts/cost-and-time.md
prompts/scrub.md
prompts/scrub-audit.md
prompts/similar-session.md
templates/case.md
templates/report.md
templates/bundle-README.md
templates/issue.md
CREATION-LOG.md
)
for rel in "${expected_files[@]}"; do
if [ -f "$SKILL_DIR/$rel" ]; then
pass "expected file present: $rel"
else
fail "expected file present: $rel"
fi
done
# --- no local paths or names in shipped files ----------------------------
leaks="$(grep -rn -E '/Users/|/home/|jesse' "$SKILL_DIR" "$SCRIPT_DIR" --exclude=test-skill-structure.sh 2>/dev/null || true)"
if [ -z "$leaks" ]; then
pass "no machine-specific paths or names in shipped files (skills + tests)"
else
fail "no machine-specific paths or names in shipped files (skills + tests)"
printf '%s\n' "$leaks" | head -10 | sed 's/^/ /'
fi
# --- "the user" never appears in skill prose -----------------------------
user_hits="$(grep -rn -i 'the user' "$SKILL_DIR" --include='*.md' 2>/dev/null | grep -v CREATION-LOG.md || true)"
if [ -z "$user_hits" ]; then
pass "skill files say 'your human partner', not 'the user'"
else
fail "skill files say 'your human partner', not 'the user'"
printf '%s\n' "$user_hits" | head -10 | sed 's/^/ /'
fi
echo
echo "Passed: $PASSES Failed: $FAILURES"
[ "$FAILURES" -eq 0 ]