Commit Graph

707 Commits

Author SHA1 Message Date
Drew Ritter 2832e01ac5 docs(plans): gate tmux ownership on creation
Keep the implementation plan's copyable tmux recipe aligned with the reviewed skill. A failed new-session now exits while ownership is false; only successful creation reaches the ownership assignment.
2026-08-06 15:05:31 -07:00
Drew Ritter 602ba17888 fix(e2e): gate tmux ownership on creation success
Make the copyable recipe independent of ambient set -e behavior. A failed tmux new-session now reports the failure and exits while session_owned is still false; only a successful creation reaches the ownership assignment.

A real collision reproduced the prior snippet killing its sentinel after marking a failed creation as owned. The guarded version preserved the exact sentinel identity, and three refined fresh-agent runs again preserved IDs 17, 18, and 19 while cleaning only their fallback sessions.
2026-08-06 15:04:25 -07:00
Drew Ritter b994fedcca fix(e2e): scope tmux cleanup to owned sessions
Replace deterministic shared tmux names and preemptive deletion with a readable run-unique name, an ownership bit set only after successful creation, and cleanup guarded by that ownership. A collision now fails closed: the runner chooses another unique name or reports failure and never reclassifies the existing session as stale.\n\nwriting-skills pressure evidence used three fresh subjects per arm with real sentinel sessions. RED was 0/3: every subject followed the old recipe by killing the pre-existing session, recreating the same name, and cleaning it again. GREEN was 3/3: every subject preserved the original session identity, created a distinct owned session, captured READY, and removed only the owned session. The controller independently verified original IDs $9, $10, and $11 before removing the disposable sentinels.
2026-08-06 14:57:57 -07:00
Drew Ritter 7dff0b2d48 fix(e2e): confine scenario card validation
Reuse the bounded fence filter for every scenario-card structural check so headings, coverage text, and falsification prose inside examples cannot satisfy the gate. Valid cards may still contain unrelated fenced examples.\n\nTreat each Card cell as a filename stem: remove at most one enclosing backtick pair, require lowercase kebab case, and reject invalid values before they can be stored or joined to the cards directory. This blocks parent traversal, subdirectories, uppercase names, and empty segments.\n\nAdd real-process RED/GREEN coverage for fenced-only cards, traversal, invalid stems, preserved backtick behavior, and the explicit diagnostic for intentionally unsupported pipe-less tables.
2026-08-06 14:50:46 -07:00
Drew Ritter 2a2e0b41c2 fix(e2e): reject fenced and malformed scenario tables
Filter backtick and tilde fenced examples before locating the E2E scenario-card table, so illustrative Markdown cannot satisfy the mechanical gate or shadow a later real table.\n\nParse only the documented leading/trailing-pipe row form and require a same-width delimiter row whose cells contain at least three hyphens with optional alignment colons. Missing or malformed delimiters now remain check failures instead of being silently treated as data.\n\nThe process-level harness records RED reproductions for both false-pass forms, fenced-only tables, and fenced examples before real tables, then verifies the focused GREEN behavior and prior contracts.
2026-08-06 14:49:13 -07:00
Drew Ritter f3a2fae88b docs(plans): implement PR 1931 safety hardening
Turn the approved bounded safety design into an executable TDD plan for the five in-scope review findings. The plan preserves canonical outer-pipe tables, defines the fenced-code and delimiter contracts, validates card filename stems before lookup, and scopes tmux cleanup to sessions created by the scenario.\n\nIt also names the RED/GREEN behavior-eval and final human-review gates while explicitly leaving browser-driving semantics and pipe-less Markdown table support out of scope. All work is rooted in the f32f review worktree based on PR head ff19e90a.
2026-08-06 14:47:11 -07:00
Drew Ritter b7b3809c8a docs(specs): complete agentic E2E scenario contract
Apply the branch's spec self-review rules to the approved safety-hardening design. Add locked E2E scenario-card rows for malformed tables, fenced structure, card-path confinement, canonical table diagnostics, and tmux ownership.

Clarify the backticked card-name boundary, set the focused tmux pressure test to three RED and three GREEN runs, and name the packaging verification command. These changes remove ambiguity without expanding the implementation scope.
2026-07-23 20:21:51 -07:00
Drew Ritter 5a1c7c58a7 docs(specs): bound agentic E2E safety hardening
Record the approved PR #1931 follow-up scope without expanding the checker into a general Markdown parser. The design keeps canonical outer-pipe tables, adds bounded fenced-code filtering, requires a delimiter row, validates card names before path construction, and gives tmux sessions explicit per-run ownership.

Define focused deterministic regression coverage and a narrowly scoped tmux RED/GREEN pressure test. Explicitly defer the browser-control and optional-outer-pipe findings so later implementation and review-thread responses cannot overclaim what this slice fixes.

Prompt context: Drew asked to spec the review feedback, avoid overworking Markdown compatibility, add safety where warranted, and defer the browser behavior change.
2026-07-23 20:20:35 -07:00
Drew Ritter ff19e90a9b docs(sdd): show the E2E offer in the example workflow
Keep the concrete subagent-driven-development example aligned with the unconditional pre-finish offer in the flowchart and normative prose. This preserves the reviewed conflict resolution after replaying the PR linearly onto current dev.
2026-07-19 15:18:41 -07:00
Jesse Vincent 3d779253aa refactor(skills): SDD e2e verification becomes pre-finish offer with disclosure doc
The predicate-keyed at-skill-start trigger is replaced by an unconditional
offer to the human after the final whole-branch review, before
finishing-a-development-branch. The procedure (spec discovery,
author/checker/runner flow, fix-wave rules) moves to spec-derived-e2e.md;
SKILL.md keeps the offer, a flowchart node, and the Integration reference.
Micro-tested 6/6: four controllers at the post-review point all offer
before finishing (including under wrap-up-efficiently pressure with the
human away); two accepted-offer controllers follow the disclosure doc with
the checker run by the controller, one fix subagent, and a fix-diff review
gate. Deviation recorded in the design spec.
2026-07-19 15:18:29 -07:00
Jesse Vincent 7fb15f0dd1 fix(skills): close checker false-pass on level-1 headings, tighten section matching 2026-07-19 15:17:52 -07:00
Jesse Vincent 37b3482931 docs(specs): record as-shipped deviations for spec-derived scenario cards 2026-07-19 15:17:52 -07:00
Jesse Vincent 0baa0498b9 feat(skills): optional spec-derived e2e verification step in SDD 2026-07-19 15:17:52 -07:00
Jesse Vincent f520b3f0de feat(skills): brainstorming specs carry E2E scenario-card tables for user-facing work 2026-07-19 15:16:56 -07:00
Jesse Vincent fdc8310a29 feat(skills): add spec-derived card authoring recipe and routing 2026-07-19 15:16:56 -07:00
Jesse Vincent 357d5028e0 fix(skills): scope card falsification match to the Expected section 2026-07-19 15:16:56 -07:00
Jesse Vincent 688b560bd4 feat(skills): add spec-vs-cards checker with test harness 2026-07-19 15:16:56 -07:00
Jesse Vincent 3eb6858469 docs(plans): add spec-derived scenario cards implementation plan 2026-07-19 15:16:56 -07:00
Jesse Vincent 158580000d docs(specs): decide card-author role boundary as flag-only 2026-07-19 15:16:56 -07:00
Jesse Vincent 1eb346b99a docs(specs): fold adversarial review findings into spec-derived scenario cards design 2026-07-19 15:16:56 -07:00
Jesse Vincent 355ae2a7e8 docs(specs): add spec-derived scenario cards design 2026-07-19 15:16:56 -07:00
Jesse Vincent a7cb9d495c fix(skills): use [DATE] placeholder in demo-movie end card
<date> inside the innerHTML string literal parses as an empty HTML
element in the browser rather than literal text if left unsubstituted.
Use the house bracket convention instead.
2026-07-19 15:16:55 -07:00
Jesse Vincent d639609808 refactor(skills): finalize e2e rationalization table and red flags 2026-07-19 15:16:55 -07:00
Jesse Vincent 2541ac5e9e feat(skills): add proof-movie and demo-movie evidence recipes 2026-07-19 15:16:55 -07:00
Jesse Vincent b6276d097c docs(skills): name concrete macOS tools in the computer-use ladder 2026-07-19 15:16:55 -07:00
Jesse Vincent f48daf84f7 feat(skills): add e2e desktop computer-use driving recipe 2026-07-19 15:16:55 -07:00
Jesse Vincent caed843a72 fix(skills): restore interleaved sleeps in TUI form-fill recipe 2026-07-19 15:16:55 -07:00
Jesse Vincent cb7b11ab2a feat(skills): add e2e browser and CLI/TUI driving recipes 2026-07-19 15:16:55 -07:00
Jesse Vincent e3d65b5c31 feat(skills): add e2e verification-runner dispatch template 2026-07-19 15:16:55 -07:00
Jesse Vincent 36d6aec57a feat(skills): add agentic-end-to-end-testing decision core 2026-07-19 15:16:55 -07:00
Jesse Vincent 2ca83e7f2a docs(plans): add agentic-end-to-end-testing implementation plan 2026-07-19 15:16:55 -07:00
Jesse Vincent 4de7d56bd6 docs(specs): add agentic-end-to-end-testing skill design 2026-07-19 15:16:55 -07:00
Jesse Vincent cc690476fc feat(sdd): lifecycle restructure with resume-based fix loop, five-round breaker, and rationalization table 2026-07-19 12:36:33 -07:00
Jesse Vincent 7ce7620d44 feat(sdd): align templates and codex reference with resume-based fix rounds 2026-07-19 12:36:33 -07:00
Jesse Vincent f428cba185 feat(sdd): add scoped re-review prompt template 2026-07-19 12:36:33 -07:00
Jesse Vincent eb1ff1f11f docs(plans): SDD fix-loop redesign implementation plan
Eight tasks across two repos: new re-review template, template/reference
alignment, full SKILL.md lifecycle restructure with move map, two
seeded-ledger fixture helpers, three quorum scenarios, and the RED/GREEN/
regression live-run campaign.
2026-07-19 12:36:33 -07:00
Jesse Vincent bea92dce1a docs(specs): SDD fix-loop redesign design spec
Review-fix loop gets resume-the-implementer semantics, scoped
re-reviews, a five-round circuit breaker, and controller adjudication
at trip. SKILL.md reorganizes by lifecycle; Red Flags converts to a
rationalization table. Brainstormed with Jesse 2026-07-15.
2026-07-19 12:36:33 -07:00
Jesse Vincent 0634449ca6 fix(tests): stop the SDD skill test flaking on timing and prose case
tests/claude-code/test-subagent-driven-development.sh failed
intermittently for two independent reasons:

- Budget mismatch: the file runs 9 prompts with a 90s timeout each
  (810s worst case) inside the runner's 600s per-file ceiling, so slow
  backend days produced spurious timeouts. Raise the runner default to
  900s and fix the help text, which claimed the default was 300.
- Case-sensitive prose matching: the assert helpers grepped free-form
  model output case-sensitively, but models capitalize the skill's own
  headings — observed failures include "Do Not Trust the Report"
  missing pattern "not trust" and a structured answer missing
  "First:.*spec.*compliance". Match case-insensitively in
  assert_contains/assert_not_contains/assert_count/assert_order, widen
  two Test 5 keyword patterns to phrasings observed in real runs, and
  make assert_order dump the output on failure the way assert_contains
  already does, so the next flake is diagnosable.

Observed 3 failures across 4 runs before the change (timeout, two
distinct pattern misses); 3/3 consecutive full runs pass after it.
2026-07-19 12:04:46 -07:00
Jesse Vincent 3fe3cb0530 fix(codex): make package script and its test portable beyond macOS/bsdtar
The packaging pipeline only worked on a Mac with default umask, for
three stacked reasons:

- The deterministic-metadata tar flags (--uid/--gid/--uname/--gname)
  are bsdtar spellings; GNU tar rejects them, so the tar.gz archive
  step died on Linux. Detect the tar flavor and use --owner=:0
  --group=:0 --numeric-owner on GNU tar, which writes byte-identical
  ustar headers (uid/gid 0, empty uname/gname).
- Staged file modes depended on two umasks canceling out: git archive
  masks entry modes with tar.umask (git default 0002 -> 775), and the
  unflagged tar extraction re-masked with the process umask (022 on
  macOS -> 755, but 002 elsewhere -> 775). Pin tar.umask=0022 on the
  archive call and extract with -p so staged modes are canonical
  755/644 on every machine.
- The test's timestamp assertion parsed bsdtar's -tv column layout and
  expected epoch 0 rendered in a US timezone ("Dec 31 1969"); GNU tar
  uses different columns and UTC hosts render "1970-01-01". Assert
  mtime == 0 via python3 tarfile instead, matching how the test
  already checks zip timestamps.

tests/codex/test-package-codex-plugin.sh now passes on Linux/GNU tar;
the bsdtar branch preserves the exact flags that passed on macOS.
2026-07-19 12:04:46 -07:00
Jesse Vincent fe0b24390e docs(windows): document shell:bash hook dispatch and the PowerShell/CMD fallback hazards 2026-07-19 12:03:59 -07:00
Jesse Vincent df78c6bfaf fix(hooks): dispatch the SessionStart hook via Git Bash on Windows
The SessionStart command string starts with a quoted path, which breaks
both Windows shells Claude Code may hand it to: PowerShell parses the
leading quoted string as an expression and dies on the next bareword
('Unexpected token session-start', #1751), and cmd.exe's /c quote rule
drops the outer quotes when the path contains a metacharacter, so a
profile dir like C:\Users\Name(External) truncates the command at the
'(' (#1918). Either way the bootstrap silently never loads.

Declare shell: "bash" on the hook. Claude Code >= 2.1.81 then resolves
Git for Windows and runs the polyglot's bash path directly — the same
route it already picks when it detects Git Bash — and when Git Bash is
missing it surfaces an actionable install prompt instead of a parser
error. Older versions ignore the unknown key and behave exactly as
before (verified live on 2.0.77 and 2.1.80).

Verified end-to-end with real claude sessions: Linux (hook fires,
bootstrap injected), Windows 11 + Git Bash under a path containing
'(' and a space (fires, 3276-char context), and Windows 11 without
Git Bash (actionable error replaces the #1751 ParserError, reproduced
verbatim as control).

Fixes #1751
Fixes #1918
2026-07-19 12:03:59 -07:00
Jesse Vincent 30ff376cb6 chore(sdd): consistency sweep for plan-scoped workspace signatures 2026-07-19 12:03:18 -07:00
Jesse Vincent 75f4e9414e eval(sdd): GREEN results — plan-scoped resolution replaces cross-plan forensics 2026-07-19 12:03:18 -07:00
Jesse Vincent c15e041e03 feat(sdd): plan-scoped durable progress — ledger names its plan, workspace dies at plan end
The start-of-skill ledger check is now scoped to the plan's own
workspace and keyed to the ledger's first line. Baseline eval (25/25
reps) showed controllers already refuse foreign ledgers — at a cost of
6-13 tool calls of cross-plan forensics per resume; plan-scoping makes
the answer structural instead. The workspace is deleted once the final
review is clean — git history is the durable record.
2026-07-19 12:03:18 -07:00
Jesse Vincent 9816a9cee2 feat(sdd): plan-scoped workspace — one .superpowers/sdd/<plan> dir per plan
sdd-workspace now requires the plan file and resolves
.superpowers/sdd/<plan-basename>/; task-brief and review-package write
into their plan's directory (review-package gains PLAN_FILE as its first
argument). Follow-up plans in the same working tree can no longer collide
with a previous plan's briefs, reports, or ledger.
2026-07-19 12:03:18 -07:00
Jesse Vincent 9d9eae52f9 eval(sdd): RED baseline — 25/25 controllers refuse stale ledgers, at a forensic cost 2026-07-19 12:03:18 -07:00
Jesse Vincent 6ddb0bfcd9 docs(specs): record eval re-scope — blind adoption did not reproduce, claims narrowed
25/25 baseline reps refused the stale foreign ledger via git forensics;
the spec's evaluation section now states the honest claims: structural
fix + measured disambiguation-cost delta + same-plan-resume regression
gate, shipping with explicit maintainer sign-off in place of a failing
S1 baseline.
2026-07-19 12:03:18 -07:00
Jesse Vincent 194907435d docs(plans): re-scope eval per maintainer decision — RED compiled, GREEN measures cost
Three RED rounds (25 reps, three framings incl. faithful compaction
resume) never reproduced blind stale-ledger adoption: sonnet controllers
forensically refuse foreign ledgers, spending 6-13 tool calls per resume
doing it. Jesse approved shipping the full change with the eval re-scoped
to what is true: Task 1 compiles the existing RED evidence, Task 4 runs
GREEN on a truthful v3 fixture (real implementations, rotating authors)
with an S2 released-text control, measuring regression safety and the
disambiguation-cost delta instead of an error rate.
2026-07-19 12:03:18 -07:00
Jesse Vincent c10431b14c docs(plans): fixture v2 — real cited commits, matched task counts
Fixture v1 tripped the Task 1 STOP gate for the right reason: its
ledgers cited fabricated hashes, so RED agents dismissed them via git
forensics (S1 passed for the wrong mechanism, the S2 resume control
failed 5/5). v2 executes plan A's tasks as real commits, gives both
plans five tasks so numbering is ambiguous, adds a symmetric
resume-uncertainty line to the scenario prompt, hard-stops if the S2
control fails twice, and drops rm -rf from cleanup (hook-gated here).
2026-07-19 12:03:18 -07:00
Jesse Vincent 0da87665c8 docs(plans): SDD plan-scoped workspace implementation plan
Five tasks: RED baseline eval (writing-skills Iron Law — before any
skill edit), plan-scoped scripts via TDD, SKILL.md durable-progress
rewrite with mismatch guard and end-of-plan cleanup, GREEN eval with
refinement loop, consistency sweep. Eval = 5 fresh sonnet subagents per
scenario per arm, hand-scored.
2026-07-19 12:03:18 -07:00