The Claude Code "human prompts" jq command counted harness-injected
user-shaped lines as typed. On a real 9,170-line transcript it returned
374 rows, one of them a <task-notification> block. Excluding text that
begins with <task-notification>, <command-name>, <local-command-stdout>,
<system-reminder>, or "This session is being continued" brings that to
76, which matches the human-turn count a careful analyst arrives at by
hand.
The same transcript carries six human prompts that never appear as user
lines at all: queued_command attachments with origin.kind "human", typed
while a turn was already running. Adds a second command for those.
Codex: on a thread_source "user" Codex Desktop rollout at cli_version
0.149.0-alpha.4.1 the user_message command returns nothing, and the
prompts arrive as response_item messages with role "user". The file
already documented that shape for subagent rollouts; it is now general,
with the fallback jq and a note that the fallback also carries injected
blocks (<subagent_notification>, <environment_context>, <skill>,
<recommended_plugins>) that have to be skipped.
Claude-Session: https://claude.ai/code/session_01DyaGKhTXvHNs2JgPhDktz7
The shipped CREATION-LOG.md quoted private repository names, issue and PR
numbers in those repos, memory-file names, and verbatim human-prompt text
from the fixture sessions. This repo is public.
The full 5,972-line record moves out of the tree and stays on the
maintainer's machine. What ships is the measurement: the method, the
scenarios with fixtures described by characteristic rather than by path,
the RED baseline results and the rationalizations the Red Flags table was
written from, the per-version table, the micro-tests with per-rep
verdicts, the four refactor rounds, a Red Flags provenance list, and the
end-to-end run.
Also surfaces the reliability finding in a Status block at the top: three
runs of scenario 12 on the same fixture and problem statement produced
three materially different substantive answers, all of which passed,
because the criteria test completeness and citation rather than
agreement. The spec now says so under Out of scope for v1, and its
report-path typo is fixed.
Claude-Session: https://claude.ai/code/session_01DyaGKhTXvHNs2JgPhDktz7
`git status --porcelain` collapses a wholly-untracked directory to a single
`?? docs/` line. In the shape of the incident this step exists for (#2016 — an
uncommitted plan document under an untracked `docs/` tree), the file list we
show the human partner therefore names no file at all:
$ git -C "$WORKTREE_PATH" status --porcelain
?? docs/
$ git -C "$WORKTREE_PATH" status --porcelain -uall
?? docs/superpowers/plans/2026-08-04-csv-export-rollout.md
Both forms produce identical (empty) output on a clean worktree, so this adds
no over-trigger surface.
Found while running this PR's behavioral micro-tests. Every treatment agent
dug past `?? docs/` unprompted and named the document, so the step did work —
but on the agent's own initiative rather than because the text asked for it.
That initiative is not reliable one tier down: Claude Haiku 4.5 on the control
arm failed for exactly this shape, asking a question that never named the file
and then deciding for the human when they deferred. Nothing in the prior
wording stopped a treatment agent from relaying `?? docs/` verbatim and
satisfying the letter of the instruction.
Re-ran the treatment cells against this amended text — Opus pass (refusal
fired, named the file), Haiku 4.5 pass (named the file) — no regression.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `dot` availability check shelled out to `which dot`, which is not a
command on Windows, so render-graphs.js reported graphviz as missing on
Windows even when it was installed. Replace it with a direct `dot -V`
probe via execFileSync.
Also switch the SVG render call from execSync to execFileSync('dot',
['-Tsvg']). Behavior is identical on macOS/Linux — the diagram source
was already passed via stdin, never interpolated into the command — but
running the binary directly removes the shell entirely.
Interrogation of reviewers who bypassed test-evidence leases showed a
convergent driver: when the report or receipt looked truncated or
couldn't be located, re-running the suite felt cheaper than re-reading —
evidence got regenerated instead of read. This paragraph names that
moment: re-read at the stated path, report a genuine gap to the
controller, and never re-run to regenerate what wasn't read.
Battery: 0/31 reviewer re-runs across 4 treatment reps vs 7/~59
reviewers in 5/8 control reps on the same scenario and classifier.
Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
Batching moves N edits under one review, which changes the review's
failure profile: an implementer that silently skips one file of twelve
produces a diff full of correct, uniform edits — nothing conspicuous is
missing, and no seat in the pipeline was assigned to notice. The single
combined review is the only net for a dropped edit, but the reviewer
template never told it to count.
The batch brief already lists every file with its change, so the reviewer
reconciles the diff against that list file by file; a listed file with no
hunk is a Missing finding regardless of how clean the rest of the batch
looks. Conditional on a multi-file brief, so single-task reviews are
unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both PRs rewrote the same preflight paragraph. Resolution is the composed
text published in #2080's description — the configuration the 3/3+3/3
composed eval grades ran.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The breaker's two ledger formats wrote lowercase 'ruling' (parked
findings, load-bearing adjudications), so the Finish section's
collect-every-`Ruling:`-line step missed exactly the rulings made under
the most pressure. Field evidence from an independent eval rep: a
breaker-cap run adjudicated correctly, wrote everything to the
plan-scoped ledger, deleted the workspace at finish, and left no durable
trace of the adjudication.
Capitalize the two breaker formats to the canonical token, and make the
finish roll-up explicitly exhaustive across preflight, parked, and
breaker rulings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
In controlled evals, an identical seeded-incoherence plan yielded 0-1/5
correct conflict resolutions when executed specless (controllers ruled
the conflicts 'internally explained') and 4-5/5 with the spec merely
present and named — even with no other skill-text changes. Cross-task
coherence turns out to be adjudicable only against ground truth above
the plan; this change makes that ground truth travel with the plan.
Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
The pre-Task-1 conflict scan currently permits 'the scan is clean' with
no evidence the scan happened — mined sessions show controllers skipping
straight to dispatch and plan conflicts surfacing mid-execution as
blocking questions. Requiring the scan to emit one row per task pair
sharing a file/interface and one row per task's self-consistency turns
the claim into an artifact; in controlled evals the table appeared 3/3
with conflicts surfaced pre-dispatch, and the mechanism held 3/3 when
composed with the never-stall ruling change (#2077).
Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
Plans sometimes enumerate many tiny, same-shape edits (one-line fixes,
constant changes, a field added across files) as separate tasks. The
current loop dispatches a fresh implementer plus review per task, so a
12-micro-task plan costs ~24 subagent seats for what one subagent could
do in a single pass. In controlled evals on a micro-task plan, batching
cut cost 73% and dispatches 87% with better completion than control; on
a 5-non-trivial-task plan the rule correctly never batched (dispatch
counts and completion identical to control).
Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
A donated session sat dormant 8h48m waiting for a plan-conflict answer
that cost ~zero tokens to decide. Wrong-ruling rework is bounded;
stalls are not. This encodes the never-stall doctrine: plan conflicts,
ambiguities, and cap exceptions get a controller ruling recorded in
the ledger and work proceeds; only irreversible/destructive actions,
security-sensitive actions, out-of-worktree side effects (merge/push/
publish), and totally-broken plans remain hard stops. Rulings surface
in the Finish report instead of as mid-run questions.
Evals: 3/3 no-stall vs control 3/3 stall-at-preflight on a
seeded-conflict SDD plan; catastrophic guard 5/5 (every rep reaching a
seeded DROP TABLE step refused it); re-validated 3/3 after rebase onto
the current fix-PR text; composes cleanly with the evidence-bearing
preflight treatment.
Claude-Session: https://claude.ai/code/session_0185AJr98gHx5EmwqNeft4Sy
Round 2 proved the long-wait mechanism (65.1%->0.0% timeouts) but
20-38 min silent waits starved graders and let 1/51 children vanish;
bounded 5-10 min stretches with a status line and list_agents
reconcile keep the efficiency and restore observability.
Docs-only wait guidance in the platform reference changed nothing
(65.1% vs 67.1% baseline wait-timeout rate); the discipline now lives
in the controller loop the session actually re-reads.
60-78% of wait_agent calls timed out across every measured corpus;
waits are event subscriptions, so one long wait replaces dozens of
polls at identical wake latency.
Triggering battery: Claude Code classified a brand-new project bounded
3/3 by reading 'existing, understood flow' as genre familiarity — once
while explicitly noting the repo was empty. Gemini routed the same
prompt architectural 3/3.
The first fix-cycle battery moved the depth-2 leak from implementers
(9/9 baseline -> 0/6) to a final reviewer that spawned two
sub-reviewers; the contract now reaches every dispatched role.
Five claims contradicted by the Codex CLI source (V2 has no
close_agent; followup_task always reaches a child; role files attach
via agent_type; full-history forks accept model/effort; V2 spawn
allowlist). Citations: superpowers-autoresearch
docs/2026-07-29-codex-multiagent-v2-capabilities.md.
Live ceremony battery: bounded reps produced zero doc ritual (the
measured win) but 2/3 implemented before any approval turn; the
bounded path now states the stop explicitly.
Spike / bounded / architectural classification said out loud, one-way
upgrade ratchet, approval gate on every path. The measured pathology:
the absolute hard-gate wording forced bounded tasks into the full
two-document ritual 5/5 while a no-guidance control differentiated
paths natively.
Rebase of PR #1922 onto current dev: the ~14 files of v6.1.0-era
codex/release drift are dropped, the porting-guide edits (stale against
the post-prune rewrite, no Hermes content) are dropped, and the Hermes
surface is kept intact: .hermes-plugin/ (on_session_start bootstrap
injection), tests/hermes/ (20 tests, passing), docs/README.hermes.md,
references/hermes-tools.md, the Platform Adaptation row, README section,
and Python ignores.
Known open items from review, unchanged by this rebase: the injection
mechanism uses ctx.inject_message from on_session_start, which the
official plugin guide does not document (pre_llm_call returning
{"context": ...} is the sanctioned path), skills are not registered via
ctx.register_skill, and the acceptance transcript predates the fix.
Co-authored-by: kumarabd <kumarabd@users.noreply.github.com>
git worktree remove refuses when the tree holds modified or untracked
files, and the skill gave no guidance for that refusal — the natural
agent response was --force, permanently destroying files that exist
nowhere else (uncommitted plans, notes, scratch work). Reported twice
from real sessions (#2016's plan loss, #1223's dirty-tree ambiguity).
Step 6 now treats the refusal as a stop-and-ask moment: show the
untracked files, offer commit / relocate / delete, and only remove the
worktree after the human partner chooses. Adds a matching rationalization
row so --force-as-cleanup is named as the failure it is.
Jesse Vincent