mirror of
https://github.com/obra/superpowers.git
synced 2026-09-01 03:19:24 +00:00
feat(diagnosing-superpowers): structure test and verified harness session references
Claude-Session: https://claude.ai/code/session_01DyaGKhTXvHNs2JgPhDktz7
This commit is contained in:
Jesse Vincent
@@ -0,0 +1,100 @@
|
||||
# Claude Code session store
|
||||
|
||||
Verified against: Claude Code 2.1.247 (transcript `version` field), macOS.
|
||||
When a field below is missing from the file in front of you, trust the file
|
||||
and say so in coverage notes.
|
||||
|
||||
## Where
|
||||
|
||||
- Main transcript: `~/.claude/projects/<cwd-slug>/<sessionId>.jsonl`, where
|
||||
`<cwd-slug>` is the working directory with every `/` replaced by `-`
|
||||
(e.g. `/tmp/work` → `-tmp-work`).
|
||||
- Subagent transcripts: `~/.claude/projects/<cwd-slug>/<sessionId>/subagents/agent-<agentId>.jsonl`,
|
||||
each with a sibling `agent-<agentId>.meta.json`
|
||||
(`agentType`, `description`, `toolUseId`, `spawnDepth`, optional `model`).
|
||||
- Plugin registry: `~/.claude/plugins/installed_plugins.json` — per plugin:
|
||||
`installPath`, `version`, `installedAt`, `lastUpdated`, `gitCommitSha`.
|
||||
- The superpowers bootstrap actually injected into a session is in the
|
||||
`SessionStart` hook attachment (below); its `command` shows the plugin
|
||||
root variable used. A dev checkout loaded with `--plugin-dir` will not be
|
||||
in the registry, so report both the registry entry and the hook evidence.
|
||||
|
||||
## Which file is the current session
|
||||
|
||||
The most recently modified `.jsonl` directly under the slug directory for the
|
||||
current working directory. Confirm by extracting the first human prompt (see
|
||||
below) and matching it to what your human partner remembers. If two files
|
||||
are close in mtime, show both first prompts and ask.
|
||||
|
||||
## Line types
|
||||
|
||||
Every line is one JSON object. `type` values seen: `user`, `assistant`,
|
||||
`attachment`, `system`, plus session-level records (`permission-mode`,
|
||||
`mode`, `bridge-session`, `last-prompt`, `ai-title`, `atis-latch`,
|
||||
`pr-link`, `queue-operation`, `relocated`, `worktree-state`).
|
||||
|
||||
Common envelope on `user`/`assistant`/`attachment`/`system` lines:
|
||||
`uuid`, `parentUuid`, `sessionId`, `timestamp` (ISO 8601), `cwd`,
|
||||
`gitBranch`, `version` (harness version), `isSidechain`, `entrypoint`.
|
||||
|
||||
| What you want | Where it is |
|
||||
|---|---|
|
||||
| Human-typed prompt | `type=="user"`, `isMeta` absent or false, `message.content` is a string or a list whose first block is `type:"text"`. Lines whose first block is `tool_result` are tool results, not prompts. `<system-reminder>` text inside a prompt is injected, not typed. |
|
||||
| Assistant text / tool calls | `type=="assistant"`, `message.content[]` blocks of `type:"text"` or `type:"tool_use"` (`id`, `name`, `input`). |
|
||||
| Tool result | `type=="user"`, `message.content[0].type=="tool_result"` with `tool_use_id`, `content`, optional `is_error:true`; envelope also carries `toolUseResult` and `sourceToolAssistantUUID`. |
|
||||
| Model | `message.model` on assistant lines. |
|
||||
| Tokens | `message.usage` on assistant lines: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`. |
|
||||
| Skill invocation | `tool_use` block with `name:"Skill"` and `input.skill` (e.g. `superpowers:brainstorming`); the tool result line has `toolUseResult.commandName`. |
|
||||
| Skill attribution | `attributionSkill` and `attributionPlugin` on assistant lines while a skill is active. |
|
||||
| Subagent dispatch | `tool_use` with `name:"Agent"` (`input.description`, `input.subagent_type`, `input.prompt`); the subagent's own file is matched by `toolUseId` in its `.meta.json`. Subagent lines have `isSidechain:true` and `agentId`. |
|
||||
| Hook output | `type=="attachment"`, `attachment.type` `hook_success`/`hook_failure`, `attachment.hookName` (e.g. `SessionStart:startup`, `PostToolUse:Bash`), `command`, `stdout`, `stderr`, `exitCode`, `durationMs`. |
|
||||
| Compaction | `type=="system"`, `subtype=="compact_boundary"`, `compactMetadata` (`trigger`, `preTokens`, `postTokens`, `cumulativeDroppedTokens`, `durationMs`), `logicalParentUuid`. |
|
||||
| Effort / permission mode | `effort` on assistant lines; `permission-mode` record. |
|
||||
|
||||
## Safe extraction
|
||||
|
||||
Lines can exceed a megabyte. Never print a whole line. Check size first:
|
||||
|
||||
```bash
|
||||
F=~/.claude/projects/<slug>/<id>.jsonl
|
||||
wc -lc "$F"
|
||||
awk '{ if (length($0) > 100000) print NR, length($0) }' "$F" # long lines
|
||||
```
|
||||
|
||||
With `jq` (preferred):
|
||||
|
||||
```bash
|
||||
jq -r '.type' "$F" | sort | uniq -c # line-type census
|
||||
jq -r 'select(.type=="user" and .isMeta!=true and ((.message.content|type)=="string" or .message.content[0].type=="text"))
|
||||
| "\(input_line_number)\t\(.timestamp)\t\((.message.content|if type=="string" then . else .[0].text end)[0:160])"' "$F" # human prompts
|
||||
jq -c 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use")
|
||||
| {name, id, input: (.input|tostring|.[0:120])}' "$F" # tool calls
|
||||
jq -r 'select(.type=="assistant") | .message.content[]? | select(.type=="tool_use" and .name=="Skill") | .input.skill' "$F" # skill invocations
|
||||
jq -c 'select(.type=="assistant") | {ts:.timestamp, model:.message.model, skill:.attributionSkill,
|
||||
u:(.message.usage|{input_tokens,output_tokens,cache_read_input_tokens,cache_creation_input_tokens})}' "$F" # per-message usage
|
||||
jq -c 'select(.subtype=="compact_boundary") | {line:input_line_number, ts:.timestamp,
|
||||
m:(.compactMetadata|{trigger,preTokens,postTokens,cumulativeDroppedTokens,durationMs})}' "$F" # compactions (full compactMetadata also has UUID lists; keep this trimmed)
|
||||
jq -c 'select(.type=="attachment" and (.attachment.type|startswith("hook"))) | {line:input_line_number, hook:.attachment.hookName, exit:.attachment.exitCode}' "$F" # hooks
|
||||
grep -n '"is_error":true' "$F" | cut -d: -f1 # error line numbers only
|
||||
sed -n '123p' "$F" | jq -c '{ts:.timestamp, first:((.message.content // "") as $c
|
||||
| ($c | if type=="array" then ($c[0] // "") else $c end) | tostring | .[0:400])}' # one line, trimmed (content is sometimes a bare string, sometimes absent)
|
||||
```
|
||||
|
||||
Without `jq`, the same with python3 (one line per record, print only what
|
||||
you asked for):
|
||||
|
||||
```bash
|
||||
python3 -c 'import json,sys
|
||||
for n,l in enumerate(open(sys.argv[1]),1):
|
||||
o=json.loads(l)
|
||||
if o.get("type")=="assistant":
|
||||
for b in o["message"].get("content",[]):
|
||||
if b.get("type")=="tool_use": print(n, b["name"], str(b.get("input"))[:120])' "$F"
|
||||
```
|
||||
|
||||
## Subagents
|
||||
|
||||
List `~/.claude/projects/<slug>/<id>/subagents/`. For each `agent-*.meta.json`
|
||||
print `agentType`, `description`, `model`; the matching `.jsonl` is that
|
||||
subagent's transcript and follows the same line format. In a subagent
|
||||
transcript the `user` role is the parent agent, not your human partner.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Codex session store
|
||||
|
||||
Verified against: Codex CLI 0.146.0 and 0.147.0 rollouts (`cli_version` in
|
||||
`session_meta`), macOS. When a field below is missing from the file in front
|
||||
of you, trust the file and say so in coverage notes.
|
||||
|
||||
## Where
|
||||
|
||||
`~/.codex/sessions/YYYY/MM/DD/rollout-<ISO-timestamp>-<thread-id>.jsonl`.
|
||||
Subagent threads are separate rollout files whose `session_meta.payload`
|
||||
has `thread_source: "subagent"` and `source.subagent.thread_spawn.parent_thread_id`
|
||||
pointing at the parent thread id. Root sessions have `thread_source: "user"`.
|
||||
|
||||
## Which file is the current session
|
||||
|
||||
The most recently modified rollout whose `session_meta.payload.cwd` is the
|
||||
current working directory and whose `thread_source` is `user`. Confirm by
|
||||
matching the first `user_message` event to what your human partner
|
||||
remembers.
|
||||
|
||||
## Line types
|
||||
|
||||
Every line is `{timestamp, type, payload}` (some also carry `ordinal`).
|
||||
`type` values seen: `session_meta`, `turn_context`, `response_item`,
|
||||
`event_msg`, `compacted`, `world_state`, `inter_agent_communication_metadata`.
|
||||
|
||||
| What you want | Where it is |
|
||||
|---|---|
|
||||
| Session identity | `session_meta.payload`: `id`, `session_id`, `cwd`, `originator` (e.g. `Codex Desktop`), `cli_version`, `model_provider`, `thread_source`, `source`, `git` (`commit_hash`, `branch`, `repository_url`), `base_instructions.text`. |
|
||||
| Model per turn | `turn_context.payload`: `turn_id`, `model`, `effort`, `cwd`, `approval_policy`, `sandbox_policy`, `multi_agent_version`. Also `event_msg` `thread_settings_applied`. |
|
||||
| Human-typed prompt | `event_msg` with `payload.type=="user_message"`: `payload.message`. (`response_item` messages with `role:"developer"` are injected, not typed; see Subagents below for rollouts that carry no `user_message` event at all.) |
|
||||
| Assistant text | `event_msg` `agent_message` (`payload.message`, `payload.phase`) or `response_item` `message` with `role:"assistant"`. |
|
||||
| Tool calls | `response_item` with `payload.type` `function_call` (`name`, `arguments`, `call_id`) or `custom_tool_call` (`name`, `input`, `call_id`); outputs are `function_call_output` / `custom_tool_call_output` matched by `call_id`. Also `event_msg` `patch_apply_end` (`success`, `changes`), `web_search_end`, `mcp_tool_call_end` (`invocation.server`, `invocation.tool`). |
|
||||
| Turn timing | `event_msg` `task_started` (`turn_id`, `started_at`, `model_context_window`) and `task_complete` (`duration_ms`, `time_to_first_token_ms`, `last_agent_message`); `turn_aborted` (`reason`, `duration_ms`). |
|
||||
| Tokens | `event_msg` `token_count`: `payload.info.total_token_usage` (cumulative; keys include `input_tokens`, `cached_input_tokens`, `output_tokens`) and `payload.rate_limits`. |
|
||||
| Compaction | a `compacted` line (`window_id`, `previous_window_id`, `replacement_history`) and an `event_msg` `context_compacted`. |
|
||||
| Subagents | `event_msg` `sub_agent_activity` (`agent_thread_id`, `agent_path`, `kind`); `response_item` `agent_message` with `author`/`recipient`; the child's own rollout file (see Where). |
|
||||
| Skill use | No attribution field. Look for `SKILL.md` in `function_call.arguments` / `custom_tool_call.input` and in `world_state`/`session_meta` instruction text. |
|
||||
| Reasoning | `response_item` `reasoning` (`summary[].text`; `encrypted_content` is opaque). |
|
||||
|
||||
## Safe extraction
|
||||
|
||||
Rollouts reach hundreds of megabytes; `compacted` lines embed whole
|
||||
histories. Never print a whole line. Check size first:
|
||||
|
||||
```bash
|
||||
F=~/.codex/sessions/YYYY/MM/DD/rollout-....jsonl
|
||||
wc -lc "$F"
|
||||
awk '{ if (length($0) > 100000) print NR, length($0) }' "$F"
|
||||
```
|
||||
|
||||
With `jq`:
|
||||
|
||||
```bash
|
||||
head -1 "$F" | jq '.payload | {id, cwd, originator, cli_version, model_provider, thread_source, git}' # identity
|
||||
jq -r '.type + "/" + (.payload.type // "")' "$F" | sort | uniq -c # census
|
||||
jq -r 'select(.type=="event_msg" and .payload.type=="user_message") | "\(input_line_number)\t\(.timestamp)\t\(.payload.message[0:160])"' "$F" # human prompts
|
||||
jq -r 'select(.type=="turn_context") | "\(.timestamp)\t\(.payload.model)\t\(.payload.effort)"' "$F" # model per turn
|
||||
jq -c 'select(.type=="response_item" and (.payload.type=="function_call" or .payload.type=="custom_tool_call"))
|
||||
| {line:input_line_number, name:.payload.name, args:((.payload.arguments // .payload.input)|tostring|.[0:120])}' "$F" # tool calls
|
||||
jq -c 'select(.payload.type=="task_complete" or .payload.type=="turn_aborted") | {ts:.timestamp, type:.payload.type, ms:.payload.duration_ms}' "$F" # turn timing
|
||||
jq -c 'select(.payload.type=="token_count") | {ts:.timestamp, t:.payload.info.total_token_usage}' "$F" # tokens (cumulative)
|
||||
grep -n '"type":"compacted"\|"context_compacted"' "$F" | cut -d: -f1 # compaction line numbers
|
||||
grep -n 'SKILL\.md' "$F" | cut -d: -f1 # skill-read line numbers
|
||||
sed -n '123p' "$F" | jq -c '{ts:.timestamp, type, p:(.payload|tostring|.[0:400])}' # one line, trimmed
|
||||
```
|
||||
|
||||
Find a thread's subagent rollouts (filenames only, never content):
|
||||
|
||||
```bash
|
||||
grep -l '"parent_thread_id":"<thread-id>"' ~/.codex/sessions/*/*/*/rollout-*.jsonl
|
||||
```
|
||||
|
||||
A subagent rollout can carry no `event_msg` `user_message` at all — the
|
||||
parent agent's dispatch prompt instead shows up as a `response_item`
|
||||
`message` with `role:"user"`. If a `user_message` event is present, it is
|
||||
from the parent agent, not your human partner.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Other harnesses: discover, then report what you found
|
||||
|
||||
This file is for any harness without a verified reference in this
|
||||
directory. You know your own harness better than this file does. Use that
|
||||
knowledge, and write down exactly what you found so the report reader can
|
||||
judge it.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Ask the harness.** Many harnesses expose a session or history command
|
||||
(`<harness> session list`, `/sessions`, a "resume" picker). Use it to get
|
||||
the session id and, if shown, the file path.
|
||||
2. **Look under the harness's config directory** (`~/.<harness>/`,
|
||||
`~/.config/<harness>/`, `~/.local/share/<harness>/`) for `sessions`,
|
||||
`history`, `chats`, `threads`, or `projects` directories holding `.jsonl`
|
||||
or `.json` files.
|
||||
3. **Confirm a candidate** by extracting its first human message with a
|
||||
size-safe command (`head -c 2000`, or `jq` on the first record) and
|
||||
matching it to what your human partner remembers. Never print whole
|
||||
lines; treat every candidate like the verified stores: `wc -lc` and a
|
||||
long-line check before anything else.
|
||||
4. **Map the fields you need** by reading a handful of records with `jq -c
|
||||
'keys'` or `head -c`: human prompt, assistant text, tool call and result,
|
||||
model, harness version, timestamps, subagent linkage, compaction.
|
||||
5. **Record in the case file and the report's coverage notes**: the store
|
||||
path, the layout you inferred, which of the fields above you could and
|
||||
could not find, and your confidence. Field-level claims in the report
|
||||
are marked "inferred from the file, not a documented format".
|
||||
6. **If you cannot find the store**, say so and ask your human partner for
|
||||
the path. Do not guess a layout from another harness.
|
||||
+128
@@ -0,0 +1,128 @@
|
||||
#!/usr/bin/env bash
|
||||
# Structural checks for skills/diagnosing-superpowers. Behavior is tested by
|
||||
# the scenarios in CREATION-LOG.md; this script only checks the things a
|
||||
# shell can check: frontmatter, referenced files exist, no local paths or
|
||||
# names leaked into shipped files, SKILL.md word budget.
|
||||
set -u
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
SKILL_DIR="$REPO_ROOT/skills/diagnosing-superpowers"
|
||||
SKILL_MD="$SKILL_DIR/SKILL.md"
|
||||
WORD_BUDGET=900
|
||||
|
||||
PASSES=0
|
||||
FAILURES=0
|
||||
|
||||
pass() { echo " [PASS] $1"; PASSES=$((PASSES + 1)); }
|
||||
fail() { echo " [FAIL] $1"; FAILURES=$((FAILURES + 1)); }
|
||||
|
||||
echo "diagnosing-superpowers structure"
|
||||
|
||||
# --- SKILL.md frontmatter -------------------------------------------------
|
||||
if [ -f "$SKILL_MD" ]; then
|
||||
pass "SKILL.md exists"
|
||||
frontmatter="$(awk 'NR==1 && $0!="---"{exit} NR>1 && $0=="---"{exit} NR>1{print}' "$SKILL_MD")"
|
||||
if printf '%s\n' "$frontmatter" | grep -q '^name: diagnosing-superpowers$'; then
|
||||
pass "frontmatter name is diagnosing-superpowers"
|
||||
else
|
||||
fail "frontmatter name is diagnosing-superpowers"
|
||||
fi
|
||||
description="$(printf '%s\n' "$frontmatter" | awk '/^description:/{sub(/^description:[ ]*/,""); print; found=1; next} found && /^[ ]/{print} found && !/^[ ]/{exit}' | tr '\n' ' ')"
|
||||
if printf '%s' "$description" | grep -q '^Use when'; then
|
||||
pass "description starts with 'Use when'"
|
||||
else
|
||||
fail "description starts with 'Use when' (got: ${description:0:60})"
|
||||
fi
|
||||
if [ "${#description}" -le 1024 ]; then
|
||||
pass "description under 1024 characters"
|
||||
else
|
||||
fail "description under 1024 characters (${#description})"
|
||||
fi
|
||||
for banned in "dispatch" "then" "step"; do
|
||||
if printf '%s' "$description" | grep -qiw "$banned"; then
|
||||
fail "description contains workflow word '$banned'"
|
||||
else
|
||||
pass "description avoids workflow word '$banned'"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- word budget --------------------------------------------------------
|
||||
body_words="$(awk 'BEGIN{fm=0} NR==1 && $0=="---"{fm=1; next} fm==1 && $0=="---"{fm=2; next} fm==2{print}' "$SKILL_MD" | wc -w | tr -d ' ')"
|
||||
if [ "$body_words" -le "$WORD_BUDGET" ]; then
|
||||
pass "SKILL.md body within $WORD_BUDGET words ($body_words)"
|
||||
else
|
||||
fail "SKILL.md body within $WORD_BUDGET words ($body_words)"
|
||||
fi
|
||||
|
||||
# --- required sections --------------------------------------------------
|
||||
for heading in "## Hard rules" "## Red Flags"; do
|
||||
if grep -q "^$heading" "$SKILL_MD"; then
|
||||
pass "SKILL.md has section '$heading'"
|
||||
else
|
||||
fail "SKILL.md has section '$heading'"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- every referenced skill file exists --------------------------------
|
||||
while IFS= read -r ref; do
|
||||
if [ -f "$SKILL_DIR/$ref" ]; then
|
||||
pass "referenced file exists: $ref"
|
||||
else
|
||||
fail "referenced file exists: $ref"
|
||||
fi
|
||||
done < <(grep -o '\(references\|prompts\|templates\)/[A-Za-z0-9._-]*\.md' "$SKILL_MD" | sort -u)
|
||||
else
|
||||
fail "SKILL.md exists"
|
||||
fi
|
||||
|
||||
# --- expected files -------------------------------------------------------
|
||||
expected_files=(
|
||||
references/claude-code-sessions.md
|
||||
references/codex-sessions.md
|
||||
references/other-harnesses.md
|
||||
prompts/skill-timeline.md
|
||||
prompts/plan-adherence.md
|
||||
prompts/repeated-work.md
|
||||
prompts/stumbles.md
|
||||
prompts/quality-evidence.md
|
||||
prompts/request-conflicts.md
|
||||
prompts/cost-and-time.md
|
||||
prompts/scrub.md
|
||||
prompts/scrub-audit.md
|
||||
prompts/similar-session.md
|
||||
templates/case.md
|
||||
templates/report.md
|
||||
templates/bundle-README.md
|
||||
templates/issue.md
|
||||
CREATION-LOG.md
|
||||
)
|
||||
for rel in "${expected_files[@]}"; do
|
||||
if [ -f "$SKILL_DIR/$rel" ]; then
|
||||
pass "expected file present: $rel"
|
||||
else
|
||||
fail "expected file present: $rel"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- no local paths or names in shipped files ----------------------------
|
||||
leaks="$(grep -rn -E '/Users/|/home/|jesse' "$SKILL_DIR" 2>/dev/null || true)"
|
||||
if [ -z "$leaks" ]; then
|
||||
pass "no machine-specific paths or names in shipped files"
|
||||
else
|
||||
fail "no machine-specific paths or names in shipped files"
|
||||
printf '%s\n' "$leaks" | head -10 | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
# --- "the user" never appears in skill prose -----------------------------
|
||||
user_hits="$(grep -rn -i 'the user' "$SKILL_DIR" --include='*.md' 2>/dev/null | grep -v CREATION-LOG.md || true)"
|
||||
if [ -z "$user_hits" ]; then
|
||||
pass "skill files say 'your human partner', not 'the user'"
|
||||
else
|
||||
fail "skill files say 'your human partner', not 'the user'"
|
||||
printf '%s\n' "$user_hits" | head -10 | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Passed: $PASSES Failed: $FAILURES"
|
||||
[ "$FAILURES" -eq 0 ]
|
||||
Reference in New Issue
Block a user