mirror/superpowers
1
0
Fork 0
mirror of https://github.com/obra/superpowers.git synced 2026-08-31 19:09:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

tests: update SDD assertions and prompts to current skill behavior

Browse Source 
The SDD skill tests still asserted on pre-rename skill text, so correct
model answers failed the suite:

- "Read at beginning" asserted "Step 1|beginning|start|Load Plan";
  the current skill has no numbered steps or Load Plan phase (setup
  covers it: "Read the plan once" during Setup). A live run today
  failed this assertion when the model correctly answered "during
  setup". Pattern now accepts setup/before-dispatch paraphrases while
  still requiring an at-the-start answer.
- "Provides text directly" asserted the removed provide-full-task-text
  behavior; SDD now routes task requirements through brief files
  (scripts/task-brief). The test now asks brief-file-vs-whole-plan and
  asserts the brief-based flow.
- The integration test's prompt and summary told the agent to
  "provide full task text to subagents (don't make them read files)",
  contradicting the skill it verifies; reworded to the task-brief flow.
  Also gave its direct `timeout 1800 claude -p` the same </dev/null
  stdin guard as run_claude.

Live-LLM tests; verified with bash -n on every touched file.

Part of #2130; defects documented in PR #2071 by @ericyen97903-lab.
This commit is contained in:
Jesse Vincent
2026-08-13 00:32:49 +00:00
parent 72ee5bbc5e
commit c7b456dd0e
2 changed files with 11 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/claude-code/test-subagent-driven-development.sh
+6 -6
View File
@@ -4,7 +4,7 @@
#
# No drill coverage: this test asks the agent to *describe* SDD (string-
# matches its verbal explanation against expected keywords like
# "self-review", "skeptical", "worktree", "Step 1", "loop"). Drill scenarios
# "self-review", "skeptical", "worktree", "setup", "loop"). Drill scenarios
# test behavior (real subagent dispatch, plan-following, review loops),
# not description-recall. Kept by design.
set -euo pipefail
@@ -83,7 +83,7 @@ else
exit 1
fi
if assert_contains "$output" "Step 1\|beginning\|start\|Load Plan" "Read at beginning"; then
if assert_contains "$output" "beginning\|start\|setup\|before.*dispatch\|before.*task" "Read at beginning"; then
: # pass
else
exit 1
@@ -133,16 +133,16 @@ echo ""
echo "Test 7: Task context provision..."
output=$(run_claude "In subagent-driven-development, how does the controller provide task information to the implementer subagent? Answer using exactly this structure:
Controller provides: <directly or by file>
Implementer must read plan file: <yes or no>" "$CLAUDE_PROMPT_TIMEOUT")
Controller provides: <brief file or whole plan file>
Implementer must read whole plan file: <yes or no>" "$CLAUDE_PROMPT_TIMEOUT")
if assert_contains "$output" "provide.*directly\|full.*text\|paste\|include.*prompt" "Provides text directly"; then
if assert_contains "$output" "task-brief\|brief file\|brief.*path\|Controller provides:.*brief" "Provides task brief file"; then
: # pass
else
exit 1
fi
if assert_contains "$output" "Implementer must read plan file:.*no" "Doesn't make subagent read file"; then
if assert_contains "$output" "Implementer must read whole plan file:.*no" "Doesn't make subagent read whole plan"; then
: # pass
else
exit 1