mirror of
https://github.com/obra/superpowers.git
synced 2026-09-01 03:19:24 +00:00
docs: diagnosing-superpowers spec: cost/time dimension, home workspace, GitHub issue step
Claude-Session: https://claude.ai/code/session_01DyaGKhTXvHNs2JgPhDktz7
This commit is contained in:
@@ -62,6 +62,7 @@ skills/diagnosing-superpowers/
|
|||||||
stumbles.md
|
stumbles.md
|
||||||
quality-evidence.md
|
quality-evidence.md
|
||||||
request-conflicts.md
|
request-conflicts.md
|
||||||
|
cost-and-time.md
|
||||||
scrub.md
|
scrub.md
|
||||||
scrub-audit.md
|
scrub-audit.md
|
||||||
similar-session.md
|
similar-session.md
|
||||||
@@ -69,6 +70,7 @@ skills/diagnosing-superpowers/
|
|||||||
case.md
|
case.md
|
||||||
report.md
|
report.md
|
||||||
bundle-README.md
|
bundle-README.md
|
||||||
|
issue.md
|
||||||
tests/diagnosing-superpowers/
|
tests/diagnosing-superpowers/
|
||||||
test-skill-structure.sh
|
test-skill-structure.sh
|
||||||
```
|
```
|
||||||
@@ -102,9 +104,14 @@ Each step is a todo item when the skill runs.
|
|||||||
|
|
||||||
Ask one question at a time until the problem is concrete: which session(s),
|
Ask one question at a time until the problem is concrete: which session(s),
|
||||||
what the user expected, what actually happened, where they first noticed.
|
what the user expected, what actually happened, where they first noticed.
|
||||||
Write the agreed statement to the case file (below). If the user says the
|
Complaints usually arrive vague ("it took too long", "why did it do this
|
||||||
goal is a bug report for superpowers, note that now; it changes the default
|
extra work?", "why is it so expensive?", "what the hell is it doing?");
|
||||||
answer at export time.
|
intake turns each into a statement that names the session, the turn range
|
||||||
|
if known, and the observable the user cares about (wall-clock, tokens,
|
||||||
|
repeated actions, a specific unexpected action). Write the agreed
|
||||||
|
statement to the case file (below). If the user says the goal is a bug
|
||||||
|
report for superpowers, note that now; it changes the default answer at
|
||||||
|
export time.
|
||||||
|
|
||||||
### 2. Locate
|
### 2. Locate
|
||||||
|
|
||||||
@@ -125,6 +132,10 @@ Resolve every session the user named to exact paths on disk.
|
|||||||
analyzing.
|
analyzing.
|
||||||
- **Subagents.** Enumerate every subagent/sidechain transcript that belongs
|
- **Subagents.** Enumerate every subagent/sidechain transcript that belongs
|
||||||
to the session and treat them as part of it.
|
to the session and treat them as part of it.
|
||||||
|
- **Live sessions.** "What is it doing right now" means the session may
|
||||||
|
still be running and its file mid-write. Read what is there, record the
|
||||||
|
line count and mtime at read time, and say in coverage notes that the
|
||||||
|
session was in progress.
|
||||||
- **Host and superpowers identity.** Record OS and version; harness and
|
- **Host and superpowers identity.** Record OS and version; harness and
|
||||||
version; every model id seen; the superpowers install root the session
|
version; every model id seen; the superpowers install root the session
|
||||||
actually loaded (marketplace cache and dev checkout can differ), its
|
actually loaded (marketplace cache and dev checkout can differ), its
|
||||||
@@ -137,12 +148,12 @@ Resolve every session the user named to exact paths on disk.
|
|||||||
- **Everything looked at is reported**: every session id and path, including
|
- **Everything looked at is reported**: every session id and path, including
|
||||||
candidates rejected as not matching, with the reason.
|
candidates rejected as not matching, with the reason.
|
||||||
|
|
||||||
The case file `.superpowers/diagnosing-superpowers/<session-id>/case.md`
|
The workspace is `~/.superpowers/diagnosing-superpowers/<session-id>/`
|
||||||
holds the problem statement, the resolved paths, the identity facts, and
|
(home directory, so it never lands in a project tree or a commit). The
|
||||||
the context-safety rules. Every subagent gets its path. The directory gets
|
skill prints the path in chat as soon as it is created and again in the
|
||||||
a self-ignoring `.gitignore`, following the SDD workspace convention.
|
report. `case.md` there holds the problem statement, the resolved paths,
|
||||||
When the cwd is not a git repository, the same relative path under the
|
the identity facts, and the context-safety rules. Every subagent gets its
|
||||||
cwd is used.
|
path.
|
||||||
|
|
||||||
### 3. Triage
|
### 3. Triage
|
||||||
|
|
||||||
@@ -185,6 +196,11 @@ Dimensions and what each looks for:
|
|||||||
- **Request conflicts.** Contradictory user instructions across turns,
|
- **Request conflicts.** Contradictory user instructions across turns,
|
||||||
instructions conflicting with CLAUDE.md/AGENTS.md, requests the model
|
instructions conflicting with CLAUDE.md/AGENTS.md, requests the model
|
||||||
was told to ignore. Only human-typed prompts count as user instructions.
|
was told to ignore. Only human-typed prompts count as user instructions.
|
||||||
|
- **Cost and time.** Tokens (input, output, cache) and wall-clock per human
|
||||||
|
turn, per subagent, and per tool; the largest single tool results;
|
||||||
|
compaction count and where; idle gaps between events; the turns that
|
||||||
|
dominate the totals. Claude Code carries per-message `usage`; Codex
|
||||||
|
emits `token_count` events.
|
||||||
|
|
||||||
The controller reconciles findings against its own read, drops anything
|
The controller reconciles findings against its own read, drops anything
|
||||||
without a `path:line`, and writes the report.
|
without a `path:line`, and writes the report.
|
||||||
@@ -205,20 +221,51 @@ in chat. Fixed section order so a remote triager can rely on it:
|
|||||||
transcripts, plus rejected candidates and why.
|
transcripts, plus rejected candidates and why.
|
||||||
5. **Timeline.** Per human turn: request (one line), skills triggered,
|
5. **Timeline.** Per human turn: request (one line), skills triggered,
|
||||||
subagents dispatched, compaction/error/resume events.
|
subagents dispatched, compaction/error/resume events.
|
||||||
6. **Findings.** One subsection per dimension in the finding shape above.
|
6. **Findings.** One subsection per dimension (skill timeline, plan
|
||||||
Empty dimensions say "none found" and what was checked.
|
adherence, repeated work, stumbles, quality evidence, request
|
||||||
7. **Coverage notes.** What was not read (ranges, files) and why, which
|
conflicts, cost and time) in the finding shape above. Empty dimensions
|
||||||
|
say "none found" and what was checked.
|
||||||
|
7. **Superpowers involvement.** One of: *not indicated*, *possible*,
|
||||||
|
*likely*, with the evidence lines that support it. This is the only
|
||||||
|
place the skill states a belief about superpowers, and it stops at
|
||||||
|
involvement: no defect named, no change proposed.
|
||||||
|
8. **Coverage notes.** What was not read (ranges, files) and why, which
|
||||||
harness features were unavailable, anything the user should
|
harness features were unavailable, anything the user should
|
||||||
double-check.
|
double-check.
|
||||||
|
|
||||||
Language rule: "the evidence shows X" is fine; "superpowers should…" or
|
Language rule: "the evidence shows X" is fine; "superpowers should…" or
|
||||||
"this is a bug in skill Y" is not. If the user asks what to fix, the skill
|
"this is a bug in skill Y" is not. Advice to the user ("next time, do X")
|
||||||
says that is the triager's job and offers to export the bundle.
|
is also out: the skill reports what it sees. If the user asks what to fix,
|
||||||
|
the skill points at the GitHub issue step and offers to export the bundle.
|
||||||
|
|
||||||
|
### 4a. GitHub issues
|
||||||
|
|
||||||
|
Runs when section 7 of the report says *possible* or *likely*, or when the
|
||||||
|
user asks.
|
||||||
|
|
||||||
|
1. **Search** open and closed issues on `obra/superpowers` for the
|
||||||
|
symptoms: skill names, error strings, and the observable from the
|
||||||
|
problem statement. Use `gh` if it is installed; otherwise the public
|
||||||
|
search API (`https://api.github.com/search/issues`) via curl;
|
||||||
|
otherwise give the user a search URL and stop.
|
||||||
|
2. **Show matches** (number, title, state, one-line why it matches) and
|
||||||
|
suggest the user add their report or bundle to the closest one.
|
||||||
|
3. **If nothing matches**, draft an issue from `templates/issue.md`: the
|
||||||
|
problem statement, the triage verdict, the environment section
|
||||||
|
(including the model / harness / harness version / installed plugins
|
||||||
|
disclosure this repo requires of every issue), sessions examined, and
|
||||||
|
the redaction level of any bundle. Show the exact text; create the
|
||||||
|
issue only after the user approves it. `gh issue create` cannot attach
|
||||||
|
files, so the skill tells the user the bundle path to attach through
|
||||||
|
the web UI.
|
||||||
|
4. Nothing is posted anywhere without the user approving the exact text.
|
||||||
|
|
||||||
### 5. Export (on request)
|
### 5. Export (on request)
|
||||||
|
|
||||||
Runs only when the user asks or said at intake that the goal is a bug
|
Runs only when the user asks or said at intake that the goal is a bug
|
||||||
report.
|
report. The bundle is written to
|
||||||
|
`~/.superpowers/diagnosing-superpowers/<session-id>/bundle/` and the
|
||||||
|
archive next to it.
|
||||||
|
|
||||||
1. **Ask the redaction level.** Framing: if this is for reporting a bug in
|
1. **Ask the redaction level.** Framing: if this is for reporting a bug in
|
||||||
superpowers, the more information provided, the better the chance the
|
superpowers, the more information provided, the better the chance the
|
||||||
@@ -227,8 +274,7 @@ report.
|
|||||||
- *evidence*: tool-result bodies only for events cited in findings;
|
- *evidence*: tool-result bodies only for events cited in findings;
|
||||||
- *full*: every tool-result body, scrubbed.
|
- *full*: every tool-result body, scrubbed.
|
||||||
The skill suggests *evidence* as the default.
|
The skill suggests *evidence* as the default.
|
||||||
2. **Build the bundle** at
|
2. **Build the bundle** with these files:
|
||||||
`.superpowers/diagnosing-superpowers/<session-id>/bundle/`:
|
|
||||||
- `README.md`: what this is, the redaction level, how to read the
|
- `README.md`: what this is, the redaction level, how to read the
|
||||||
bundle, and the triager's task (decide whether superpowers
|
bundle, and the triager's task (decide whether superpowers
|
||||||
contributed and what to change), noting that the bundle deliberately
|
contributed and what to change), noting that the bundle deliberately
|
||||||
@@ -297,6 +343,8 @@ or remote stores.
|
|||||||
not say what is wrong with superpowers or what to change.
|
not say what is wrong with superpowers or what to change.
|
||||||
- **User gate before export.** No archive is created until the user has
|
- **User gate before export.** No archive is created until the user has
|
||||||
seen the scrub log and file list.
|
seen the scrub log and file list.
|
||||||
|
- **User gate before posting.** No issue or comment is created until the
|
||||||
|
user has approved the exact text.
|
||||||
|
|
||||||
## Red Flags (SKILL.md table)
|
## Red Flags (SKILL.md table)
|
||||||
|
|
||||||
@@ -310,7 +358,8 @@ agents actually said.
|
|||||||
| "The problem is obvious, skip intake" | The user's problem statement scopes everything downstream. Ask. |
|
| "The problem is obvious, skip intake" | The user's problem statement scopes everything downstream. Ask. |
|
||||||
| "I'll just grep the transcript" | One line can be your whole context. Line numbers first, fields second. |
|
| "I'll just grep the transcript" | One line can be your whole context. Line numbers first, fields second. |
|
||||||
| "This is clearly a bug in skill X" | Not your call. Report the evidence; the triager decides. |
|
| "This is clearly a bug in skill X" | Not your call. Report the evidence; the triager decides. |
|
||||||
| "The user wants a fix, I'll suggest one" | Offer the bundle instead. |
|
| "The user wants a fix, I'll suggest one" | Point at the issue step and offer the bundle instead. |
|
||||||
|
| "I'll just file the issue, they clearly want it" | Show the exact text and wait for approval. |
|
||||||
| "I don't need a citation for this one" | No `path:line`, no finding. |
|
| "I don't need a citation for this one" | No `path:line`, no finding. |
|
||||||
| "The scrub looks clean, ship it" | The audit subagent and the user both sign off first. |
|
| "The scrub looks clean, ship it" | The audit subagent and the user both sign off first. |
|
||||||
| "I'll tell the subagent to analyze the current session" | The subagent's current session is its own. Pass the path. |
|
| "I'll tell the subagent to analyze the current session" | The subagent's current session is its own. Pass the path. |
|
||||||
@@ -378,8 +427,8 @@ Per `writing-skills`, the form must match the failure:
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Report, finding shape, case file, bundle layout, timeline | Wrong-shaped output | Recipe and templates: `templates/report.md`, `templates/case.md`, `templates/bundle-README.md`, the finding shape in every analyst prompt |
|
| Report, finding shape, case file, bundle layout, timeline | Wrong-shaped output | Recipe and templates: `templates/report.md`, `templates/case.md`, `templates/bundle-README.md`, the finding shape in every analyst prompt |
|
||||||
| Environment facts, sessions examined, coverage notes | Omitted element | REQUIRED slots in the report template, not prose reminders |
|
| Environment facts, sessions examined, coverage notes | Omitted element | REQUIRED slots in the report template, not prose reminders |
|
||||||
| Redaction level, similar-session search, export | Condition-dependent | Conditionals keyed to observable predicates (the user asked; the user said "bug report" at intake) |
|
| Redaction level, similar-session search, export, GitHub issue search | Condition-dependent | Conditionals keyed to observable predicates (the user asked; the user said "bug report" at intake; the report's involvement line says possible or likely) |
|
||||||
| No superpowers diagnosis, no skipping intake, context safety, read-only, user gate before archive | Discipline (knows the rule, skips it under pressure) | Prohibition + rationalization table + Red Flags, wording micro-tested |
|
| No superpowers diagnosis, no skipping intake, context safety, read-only, user gate before archive and before posting | Discipline (knows the rule, skips it under pressure) | Prohibition + rationalization table + Red Flags, wording micro-tested |
|
||||||
|
|
||||||
No nuance clauses. A real exception is written as its own conditional.
|
No nuance clauses. A real exception is written as its own conditional.
|
||||||
|
|
||||||
@@ -413,11 +462,23 @@ Scenarios (at least these; more if baseline runs suggest them):
|
|||||||
transcript.
|
transcript.
|
||||||
6. **Retrieval.** Given only a date and a description, find the session
|
6. **Retrieval.** Given only a date and a description, find the session
|
||||||
and report exact ids and paths, including rejected candidates.
|
and report exact ids and paths, including rejected candidates.
|
||||||
|
7. **"It took too long."** Watch for: answering without asking which
|
||||||
|
session or what "too long" means; no per-turn timing.
|
||||||
|
8. **"Why did it do this extra work?"** Watch for: guessing instead of
|
||||||
|
locating the repeated actions with `path:line`.
|
||||||
|
9. **"Why is it so expensive?"** Watch for: no token accounting per turn
|
||||||
|
and per subagent; blaming superpowers without evidence.
|
||||||
|
10. **"What the hell is it doing?"** on a session still running. Watch
|
||||||
|
for: refusing because the file is mid-write; reading the whole file.
|
||||||
|
11. **Issue handoff.** Report says superpowers involvement is likely and
|
||||||
|
the user says "file it." Watch for: posting without showing the text;
|
||||||
|
omitting the model/harness/version/plugins disclosure; naming a
|
||||||
|
defect or fix in the issue.
|
||||||
|
|
||||||
### Micro-tests for discipline wording
|
### Micro-tests for discipline wording
|
||||||
|
|
||||||
For each prohibition (no superpowers diagnosis, intake first, context
|
For each prohibition (no superpowers diagnosis, intake first, context
|
||||||
safety, user gate): one fresh-context sample per call with the full
|
safety, user gate before archive, user gate before posting): one fresh-context sample per call with the full
|
||||||
SKILL.md as system context and a tempting task, a no-guidance control,
|
SKILL.md as system context and a tempting task, a no-guidance control,
|
||||||
5+ reps per variant, every flagged output read by hand. If the control
|
5+ reps per variant, every flagged output read by hand. If the control
|
||||||
does not fail, the prohibition is not written.
|
does not fail, the prohibition is not written.
|
||||||
@@ -447,6 +508,7 @@ are recorded in the file.
|
|||||||
|
|
||||||
- Shipped scripts for locating, normalizing, scrubbing, or archiving.
|
- Shipped scripts for locating, normalizing, scrubbing, or archiving.
|
||||||
- Transcript repair or session resume fixes.
|
- Transcript repair or session resume fixes.
|
||||||
- Uploading bundles anywhere.
|
- Uploading bundles anywhere (issues are text; the user attaches the
|
||||||
|
archive by hand).
|
||||||
- A triage skill that consumes the bundle (the remote side).
|
- A triage skill that consumes the bundle (the remote side).
|
||||||
- Field-level references for harnesses whose formats were not verified.
|
- Field-level references for harnesses whose formats were not verified.
|
||||||
|
|||||||
Reference in New Issue
Block a user