docs: design spec for diagnosing-superpowers skill

Claude-Session: https://claude.ai/code/session_01DyaGKhTXvHNs2JgPhDktz7

docs/superpowers/specs/2026-08-27-diagnosing-superpowers-design.md

@@ -0,0 +1,387 @@
+# Diagnosing Superpowers Sessions — Design
+
+Date: 2026-08-27
+Status: approved by Jesse (in-session); spec pending review
+Branch: `diagnosing-superpowers` off `dev`
+
+## Goal
+
+A core skill, `diagnosing-superpowers`, that a user invokes when a
+superpowers session went wrong. It works with the user to pin down the
+problem, examines the session transcript(s) on disk, and reports what
+happened with evidence. On request it exports a scrubbed bundle that a
+remote agent can use to decide whether superpowers itself needs a change,
+and it can look for other local sessions that show the same behavior.
+
+The skill reports; it never diagnoses superpowers. Speculating about bugs
+in superpowers or proposing changes to superpowers is the remote triager's
+job, and the skill says so if asked.
+
+## Scope decisions (settled with Jesse)
+
+- **Pure prose skill for v1.** No shipped scripts. The model does the work,
+  using subagents aggressively. Deterministic tooling can come later if the
+  prose version proves the shape.
+- **Harness coverage.** Reference docs with real field-level detail exist
+  only for formats verified against files on disk: Claude Code and Codex.
+  Every other harness gets a discovery procedure. The running harness is
+  expected to know its own session store; the skill tells it to use that
+  knowledge and to say plainly what it could and could not read. No
+  invented formats.
+- **Problem intake first.** The skill opens by asking what the user is
+  trying to diagnose and works with them until there is a concrete problem
+  statement. Sweeps run in service of that statement.
+- **Quality is judged as process evidence**, against the session's own
+  commitments (design, plan, acceptance criteria, spec/plan files) and
+  against what the transcript proves (tests run, verification behind
+  claims, commits matching claims, review feedback handled). It is not a
+  code review of the resulting diff.
+- **Redaction level is the user's call.** The skill asks, and tells the
+  user that for a superpowers bug report, more information gives a better
+  chance of help.
+- **Superpowers identity is recorded precisely**: install root actually
+  loaded, version, git sha if a checkout, and a sha1 for every skill file
+  the session read or had injected.
+- **Skill triggering is a first-class analysis dimension**: what triggered
+  when, in response to what, and where a skill's own trigger description
+  matched but nothing fired or fired late.
+
+## Skill layout
+
+```
+skills/diagnosing-superpowers/
+  SKILL.md
+  references/
+    claude-code-sessions.md
+    codex-sessions.md
+    other-harnesses.md
+  prompts/
+    skill-timeline.md
+    plan-adherence.md
+    repeated-work.md
+    stumbles.md
+    quality-evidence.md
+    request-conflicts.md
+    scrub.md
+    scrub-audit.md
+    similar-session.md
+  templates/
+    case.md
+    report.md
+    bundle-README.md
+tests/diagnosing-superpowers/
+  test-skill-structure.sh
+```
+
+Same shape as `subagent-driven-development`: a lean SKILL.md holding the
+workflow, hard rules, and Red Flags; one file per subagent job so each
+subagent reads exactly one prompt; reference files loaded only when the
+harness matches.
+
+### SKILL.md frontmatter
+
+```
+name: diagnosing-superpowers
+description: Use when a superpowers session went wrong and the user wants
+  to know why — repeated work, ignored plans, stumbles, poor results, a
+  skill that didn't fire — or wants to build a bug report for the
+  superpowers maintainers. Works on the current session or a past one by
+  id or path, on any harness.
+```
+
+## Workflow
+
+Each step is a todo item when the skill runs.
+
+### 1. Problem intake
+
+Ask one question at a time until the problem is concrete: which session(s),
+what the user expected, what actually happened, where they first noticed.
+Write the agreed statement to the case file (below). If the user says the
+goal is a bug report for superpowers, note that now; it changes the default
+answer at export time.
+
+### 2. Locate
+
+Resolve every session the user named to exact paths on disk.
+
+- **Current session.** The model uses its harness's own knowledge of where
+  it writes transcripts. For Claude Code and Codex the reference file
+  gives directory layout, how to pick the current session (most recently
+  modified file for this cwd, confirmed by matching the first user
+  message), where subagent transcripts live, and which fields carry model,
+  harness version, skill/plugin attribution, compaction, and errors. For
+  any other harness, `other-harnesses.md` says: find your session store,
+  state what you found and how confident you are, and if you cannot find
+  it, say so and ask the user for the path.
+- **Past session.** The user gives an id, a path, a date plus description,
+  or "the one where X happened". Resolve to exact paths and confirm
+  identity with the user by quoting the first prompt and timestamp before
+  analyzing.
+- **Subagents.** Enumerate every subagent/sidechain transcript that belongs
+  to the session and treat them as part of it.
+- **Host and superpowers identity.** Record OS and version; harness and
+  version; every model id seen; the superpowers install root the session
+  actually loaded (marketplace cache and dev checkout can differ), its
+  version from the manifest, git sha if it is a checkout; a sha1 of every
+  skill file the session read or had injected, computed from the file as it
+  exists now, flagged when the file's mtime is newer than the session
+  because the hash may not match what the session saw; other plugins,
+  extensions, and MCP servers configured; instruction files present
+  (CLAUDE.md, AGENTS.md, GEMINI.md, and the like) listed by path only.
+- **Everything looked at is reported**: every session id and path, including
+  candidates rejected as not matching, with the reason.
+
+The case file `.superpowers/diagnosing-superpowers/<session-id>/case.md`
+holds the problem statement, the resolved paths, the identity facts, and
+the context-safety rules. Every subagent gets its path. The directory gets
+a self-ignoring `.gitignore`, following the SDD workspace convention.
+When the cwd is not a git repository, the same relative path under the
+cwd is used.
+
+### 3. Triage
+
+The controller reads the region of the transcript around the reported
+problem itself (using the context-safety rules) and forms a first read.
+Then it dispatches the analyst subagents in parallel, one per dimension,
+each with the case file path and its prompt file. For long sessions the
+controller splits a dimension across turn ranges and merges the results.
+
+Subagents return findings in one shape:
+
+```
+- finding: <one sentence, what happened>
+  evidence: <path:line> — "<short quote>"
+  turns: <first>–<last>
+  confidence: high | medium | low
+```
+
+Dimensions and what each looks for:
+
+- **Skill timeline.** Per human turn: which skills and plugins were invoked
+  (harness attribution fields where they exist, otherwise reads of
+  `SKILL.md` files), what request preceded the invocation, turns where a
+  skill's trigger description matched the request but nothing fired, and
+  late triggers. Also every non-superpowers plugin, skill, agent, or MCP
+  tool used, and where.
+- **Plan adherence.** Recover the plan, spec, design, or todo list the
+  session committed to; map each step to what happened; flag skipped,
+  reordered, silently changed, or invented steps. Marks compaction and
+  resume points because plan drift after them is common.
+- **Repeated work.** Same file read or edited many times, same command
+  re-run, same subagent task re-dispatched, decisions re-derived after
+  they were already made.
+- **Stumbles.** Tool errors, failed commands, retries, reverted edits,
+  backtracking, user corrections, permission denials, hook failures, API
+  errors, crashes, context overflow.
+- **Quality evidence.** Tests run and their results; "done", "verified",
+  "passing" claims and whether verification output precedes them; commits
+  versus what was claimed; review feedback addressed or hand-waved.
+- **Request conflicts.** Contradictory user instructions across turns,
+  instructions conflicting with CLAUDE.md/AGENTS.md, requests the model
+  was told to ignore. Only human-typed prompts count as user instructions.
+
+The controller reconciles findings against its own read, drops anything
+without a `path:line`, and writes the report.
+
+### 4. Report
+
+`.superpowers/diagnosing-superpowers/<session-id>/report.md`, also shown
+in chat. Fixed section order so a remote triager can rely on it:
+
+1. **Problem statement** as agreed at intake.
+2. **Triage verdict.** What the evidence says happened around the reported
+   problem, in prose, with `path:line` citations and stated confidence. No
+   root-cause claims about superpowers and no recommendations for it.
+3. **Environment.** Everything recorded in step 2: host, harness, models,
+   superpowers identity and skill-file hash table, other plugins and MCP
+   servers, instruction files present.
+4. **Sessions examined.** Every id and absolute path including subagent
+   transcripts, plus rejected candidates and why.
+5. **Timeline.** Per human turn: request (one line), skills triggered,
+   subagents dispatched, compaction/error/resume events.
+6. **Findings.** One subsection per dimension in the finding shape above.
+   Empty dimensions say "none found" and what was checked.
+7. **Coverage notes.** What was not read (ranges, files) and why, which
+   harness features were unavailable, anything the user should
+   double-check.
+
+Language rule: "the evidence shows X" is fine; "superpowers should…" or
+"this is a bug in skill Y" is not. If the user asks what to fix, the skill
+says that is the triager's job and offers to export the bundle.
+
+### 5. Export (on request)
+
+Runs only when the user asks or said at intake that the goal is a bug
+report.
+
+1. **Ask the redaction level.** Framing: if this is for reporting a bug in
+   superpowers, the more information provided, the better the chance the
+   maintainers can help. Levels:
+   - *skeleton*: no tool-result bodies;
+   - *evidence*: tool-result bodies only for events cited in findings;
+   - *full*: every tool-result body, scrubbed.
+   The skill suggests *evidence* as the default.
+2. **Build the bundle** at
+   `.superpowers/diagnosing-superpowers/<session-id>/bundle/`:
+   - `README.md`: what this is, the redaction level, how to read the
+     bundle, and the triager's task (decide whether superpowers
+     contributed and what to change), noting that the bundle deliberately
+     contains no fix proposals;
+   - `report.md`, `case.md`, `environment.json`, `timeline.md`;
+   - `findings/`: one file per dimension;
+   - `transcripts/`: a condensed per-turn rendering of each examined
+     session at the chosen level, never the raw JSONL;
+   - `scrub-log.md`.
+3. **Scrub** by subagent, per file: emails; names of people, replaced with
+   role placeholders; account and organization UUIDs; anything that looks
+   like an API key, token, or password; hostnames and IPs; absolute paths
+   under home rewritten to `~`; repository names and URLs unless the user
+   says the repository is public; anything the user names as proprietary.
+   Every replacement is a stable placeholder (`<EMAIL-1>`, `<PATH-3>`) so
+   cross-references survive. The scrub log lists placeholder → category,
+   never the original value.
+4. **Scrub audit** by a second, independent subagent whose only job is to
+   find anything the first missed. Repeat scrub and audit until the audit
+   finds nothing.
+5. **User review gate.** Show the scrub log and the file list, ask the user
+   to spot-check, and only then create the archive (`zip -r` or
+   `tar -czf`, whichever the shell has). Report the archive path. The skill
+   never uploads anything anywhere.
+
+### 6. Similar sessions (on request)
+
+1. Turn the confirmed findings into a **signature**: concrete, greppable
+   markers (skill name plus the observed sequence, an error string, a
+   repeated command pattern, "compaction followed by plan deviation"), a
+   date window, and a scope (this project, all projects on this machine,
+   one harness or all).
+2. Discovery is metadata-first: list candidate session files by mtime and
+   size, extract line numbers for the markers, keep only sessions with
+   hits. Context-safety rules apply.
+3. Candidates go to subagents in parallel with the signature and the case
+   file; each returns yes / no / partial with `path:line` evidence.
+4. Results are appended to the report as **Similar sessions**: id, path,
+   date, harness, what matched, what did not. Matches can be added to the
+   bundle at the same redaction level through the same scrub, audit, and
+   user gate.
+
+Local machine only. The skill never reaches into other people's sessions
+or remote stores.
+
+## Hard rules (SKILL.md and every subagent prompt)
+
+- **Context safety.** Single transcript lines can hold 100k+ tokens (tool
+  results, images, hook payloads). Never `cat` or `grep` a transcript for
+  content. Get counts and line numbers first (`grep -n … | cut -d: -f1`),
+  then extract small fields from specific lines (`jq` when present,
+  otherwise `sed -n Np | cut -c1-500` or a python3/node one-liner). Check
+  the file size and line count before anything else.
+- **Read-only.** Session files are never modified, moved, or deleted.
+- **Exact paths to subagents.** "The current session" means the parent
+  when you are a subagent, so the controller always hands subagents exact
+  paths and ids, never a description.
+- **Human prompts only.** Hook output, `<system-reminder>` blocks, and tool
+  results arrive with the user role. Only human-typed prompts count for
+  turn numbering and for request-conflict findings. In a subagent
+  transcript, "user" is the parent agent.
+- **Evidence or nothing.** Every finding cites `path:line`. Findings without
+  a citation are dropped at reconciliation.
+- **No superpowers diagnosis.** The skill describes what happened. It does
+  not say what is wrong with superpowers or what to change.
+- **User gate before export.** No archive is created until the user has
+  seen the scrub log and file list.
+
+## Red Flags (SKILL.md table)
+
+| Thought | Reality |
+|---------|---------|
+| "The problem is obvious, skip intake" | The user's problem statement scopes everything downstream. Ask. |
+| "I'll just grep the transcript" | One line can be your whole context. Line numbers first, fields second. |
+| "This is clearly a bug in skill X" | Not your call. Report the evidence; the triager decides. |
+| "The user wants a fix, I'll suggest one" | Offer the bundle instead. |
+| "I don't need a citation for this one" | No `path:line`, no finding. |
+| "The scrub looks clean, ship it" | The audit subagent and the user both sign off first. |
+| "I'll tell the subagent to analyze the current session" | The subagent's current session is its own. Pass the path. |
+| "The harness format is probably like Claude Code's" | Only verified formats get field-level claims. Discover, then report what you found. |
+
+## Harness reference files
+
+### `references/claude-code-sessions.md`
+
+Verified against files on this machine, Claude Code 2.1.247:
+
+- Store: `~/.claude/projects/<cwd-slug>/<sessionId>.jsonl` where the slug
+  is the cwd with `/` replaced by `-`.
+- Subagents: `~/.claude/projects/<cwd-slug>/<sessionId>/subagents/agent-<id>.jsonl`
+  with a sibling `agent-<id>.meta.json`.
+- Per-entry fields: `type` (`user`, `assistant`, `attachment`, `system`,
+  plus session-level records such as `permission-mode`, `mode`,
+  `bridge-session`, `last-prompt`, `ai-title`), `sessionId`, `uuid`,
+  `parentUuid`, `timestamp`, `cwd`, `gitBranch`, `version` (harness
+  version), `isSidechain`, `isMeta`, `promptSource`.
+- Assistant entries: `message.model`, `attributionSkill`,
+  `attributionPlugin`, `requestId`, `effort`.
+- Compaction: `system` entries with `subtype: compact_boundary`.
+- Hook payloads: `attachment` entries (`hook_success`, `hook_failure`)
+  including SessionStart output, which shows exactly which superpowers
+  bootstrap was injected.
+- Plugin registry: `~/.claude/plugins/installed_plugins.json`
+  (`installPath`, `version`, `gitCommitSha` per plugin). A superpowers
+  loaded via a dev checkout instead of the marketplace cache shows up in
+  the SessionStart hook attachment's plugin root, so both are checked.
+
+### `references/codex-sessions.md`
+
+Verified against files on this machine, Codex CLI 0.147.0:
+
+- Store: `~/.codex/sessions/YYYY/MM/DD/rollout-<timestamp>-<id>.jsonl`.
+- `session_meta` line: `payload.id`, `payload.session_id`,
+  `payload.parent_thread_id`, `payload.cwd`, `payload.originator`,
+  `payload.cli_version`, `payload.model_provider`, `payload.source`
+  (subagent spawn details: `parent_thread_id`, `depth`, `agent_nickname`).
+  Subagent rollouts are separate files linked by `parent_thread_id`.
+- Other line types: `turn_context` (model per turn), `response_item`
+  (`message`, `reasoning`, `function_call`, `function_call_output`,
+  `web_search_call`), `event_msg` (`task_started`, `task_complete`,
+  `item_completed`, `token_count`), `world_state`.
+- No skill attribution field. Skill use is inferred from
+  `function_call` reads of `SKILL.md` paths and from the multi-agent
+  spawn records.
+
+### `references/other-harnesses.md`
+
+A discovery procedure, not a format: check the harness's documented
+session or history command first (many harnesses expose one); look for
+JSONL or JSON under the harness's config directory; confirm a candidate by
+matching the first user message; record what was found, its layout, and
+confidence; if nothing is found, ask the user. Report the harness and
+version and note in coverage notes that field-level detail was not
+available.
+
+## Testing
+
+- **Skill behavior** per `writing-skills`: pressure-test with a subagent
+  given a real local transcript and the skill, checked for: asked about
+  the problem before analyzing; refused to propose superpowers fixes when
+  pushed; cited paths, ids, and `path:line`; followed context-safety rules
+  on a transcript with a multi-megabyte line; stopped for the user gate
+  before archiving. Scenarios and results go in
+  `skills/diagnosing-superpowers/CREATION-LOG.md`, following the
+  `systematic-debugging` precedent.
+- **Structure** in `tests/diagnosing-superpowers/test-skill-structure.sh`:
+  frontmatter present and well-formed; every prompt, reference, and
+  template file referenced from SKILL.md exists; no machine-specific
+  absolute paths or user names baked into shipped files; the Red Flags
+  table and hard rules sections exist.
+- Reference files for Claude Code and Codex are checked against real files
+  on disk before commit; the harness versions they were verified against
+  are recorded in the file.
+
+## Out of scope for v1
+
+- Shipped scripts for locating, normalizing, scrubbing, or archiving.
+- Transcript repair or session resume fixes.
+- Uploading bundles anywhere.
+- A triage skill that consumes the bundle (the remote side).
+- Field-level references for harnesses whose formats were not verified.