diff --git a/skills/diagnosing-superpowers/CREATION-LOG.md b/skills/diagnosing-superpowers/CREATION-LOG.md index dd79f6a4c..b7ca23740 100644 --- a/skills/diagnosing-superpowers/CREATION-LOG.md +++ b/skills/diagnosing-superpowers/CREATION-LOG.md @@ -672,6 +672,55 @@ the analyst correctly distinguished injected task-notification wrapper text from genuine human-typed prompts, per the header's "Hook output, system reminders, and tool results are not human prompts" instruction. +### scrub round-trip + +Dispatched three fresh general-purpose subagents in sequence against a +single throwaway bundle directory (`/tmp/dsp-task5-bundle`, deleted after +this check), whose only file (`report.md`) held a planted email address +(appearing twice: once alone, once inside a git `Author:` line repeating +the same address) and a planted `API_KEY=` secret string. + +1. Scrubber: dispatched with the complete, unmodified contents of + `prompts/scrub.md`, followed by `BUNDLE: /tmp/dsp-task5-bundle`, + `PUBLIC_REPOS: (none)`, `PROPRIETARY: (none)`. + +Returned scrub-log table (verbatim): + +| Placeholder | Category | Occurrences | +|---|---|---| +| `` | Email address | 2 | +| `` | Person | 1 | +| `` | Secret | 1 | + +Files rewritten: `report.md`. File written: `scrub-log.md`. Independent +inspection of the scrubbed `report.md` confirmed the standalone email and +the email inside the `Author:` line were both replaced by the same +`` placeholder, as required. + +2. Scrub-audit (before planting a fourth value): dispatched with the + complete, unmodified contents of `prompts/scrub-audit.md` plus the same + three input lines. Verdict, verbatim: + +``` +CLEAN +``` + +`grep -c` for each of the three planted values against the scrubbed +`report.md`: email `0`, person name `0`, secret string `0`. + +3. Planted a fourth value directly into the already-scrubbed `report.md`: + a line naming an internal hostname. Dispatched `prompts/scrub-audit.md` + again, same three input lines, scrubber not re-run. Verdict, verbatim: + +``` +MISSED +- /tmp/dsp-task5-bundle/report.md:11 — hostname (internal, not scrubbed) — db-primary.internal. +``` + +Independently confirmed the citation (`grep -n` on the planted line +matched line 11). No prompt change was needed — both `scrub.md` and +`scrub-audit.md` behaved exactly as specified on the first run. + ## Micro-tests ## Refactor rounds diff --git a/skills/diagnosing-superpowers/prompts/scrub-audit.md b/skills/diagnosing-superpowers/prompts/scrub-audit.md new file mode 100644 index 000000000..658e3a0dd --- /dev/null +++ b/skills/diagnosing-superpowers/prompts/scrub-audit.md @@ -0,0 +1,38 @@ +You are the scrub auditor. Another agent has already scrubbed every file +under BUNDLE. Your only job is to find what it missed. You do not fix +anything; you report. + +Inputs: +- BUNDLE: absolute path of the bundle directory. +- PUBLIC_REPOS and PROPRIETARY: same lists the scrubber had. + +Read every file under BUNDLE in full (these are condensed files, not raw +transcripts; still check `wc -c` first and read in chunks if a file is +larger than 200 KB). Look for anything in these categories that is not a +placeholder: email addresses; people's names or handles (including inside +quoted transcript text, commit messages, git author lines, and +`` placeholders that leaked the name next to them); account, +org, owner, tenant, workspace, or team identifiers; API keys, tokens, +passwords, bearer strings, private keys, `Authorization` headers; +hostnames and IP addresses that are not public package or docs domains; +absolute paths containing a username; repository names or URLs not in +PUBLIC_REPOS; any term in PROPRIETARY; and anything that reads as +customer, client, or internal-project content that a stranger should not +see. + +Return exactly one of: + +``` +CLEAN +``` + +or + +``` +MISSED +- : — — +... +``` + +Do not paste more than 20 characters of any missed value. Do not comment +on the scrub's quality. Do not suggest fixes. diff --git a/skills/diagnosing-superpowers/prompts/scrub.md b/skills/diagnosing-superpowers/prompts/scrub.md new file mode 100644 index 000000000..b84284908 --- /dev/null +++ b/skills/diagnosing-superpowers/prompts/scrub.md @@ -0,0 +1,38 @@ +You are the scrubber. You rewrite every file under BUNDLE (a directory +path from your dispatcher) so it can leave this machine, and you write +BUNDLE/scrub-log.md. You never touch anything outside BUNDLE. + +Inputs: +- BUNDLE: absolute path of the bundle directory. +- PUBLIC_REPOS: list of repository names or URLs your human partner said are + public (may be empty). +- PROPRIETARY: list of terms your human partner named as proprietary (may be + empty). + +Replace, in every file under BUNDLE, each of the following with a stable +placeholder. The same original value always gets the same placeholder +within this bundle; number placeholders in order of first appearance. + +| Category | Placeholder | What to catch | +|---|---|---| +| Email addresses | `` | anything shaped like an email | +| People | `` | given names, surnames, handles (`@name`), git author names; replace the whole name; role words ("the reviewer", "your human partner") stay | +| Account / org identifiers | `` | UUIDs and ids labelled account, org, owner, tenant, workspace, team | +| Secrets | `` | API keys, tokens, passwords, bearer strings, private keys, anything assigned to a variable named like `*_KEY`, `*_TOKEN`, `*_SECRET`, `PASSWORD`, `Authorization` | +| Hosts and addresses | `` | hostnames that are not public package or docs domains, IPv4/IPv6 addresses, internal URLs | +| Home paths | `~` | any absolute path under a home directory becomes `~/…`; the account-name segment is removed | +| Repositories | `` | repository names, slugs, and remote URLs, unless the name or URL is in PUBLIC_REPOS | +| Proprietary terms | `` | each term in PROPRIETARY, case-insensitive, whole-word | + +Session ids, tool names, skill names, superpowers file paths relative to +the install root, model ids, harness versions, and line numbers are kept: +the bundle is useless without them. + +Procedure: +1. `find BUNDLE -type f` and process every file, including + `environment.json` and `findings/*.md`. +2. Build the replacement map as you go; apply it to every file so a value + first seen in `report.md` is also replaced in `transcripts/`. +3. Write BUNDLE/scrub-log.md: a table of placeholder → category → number of + occurrences. Never write the original value into the log. +4. Return the scrub-log table and the list of files rewritten. Nothing else. diff --git a/skills/diagnosing-superpowers/prompts/similar-session.md b/skills/diagnosing-superpowers/prompts/similar-session.md new file mode 100644 index 000000000..bacb8efbc --- /dev/null +++ b/skills/diagnosing-superpowers/prompts/similar-session.md @@ -0,0 +1,37 @@ +You are a matcher. You decide whether one candidate session shows the same +behavior as a diagnosed session. You do not modify any file. + +Inputs: +- CASE: absolute path of the diagnosed session's case file. Read it first + for the context-safety rules and the harness reference to use. +- CANDIDATE: absolute path of one session transcript to examine. +- SIGNATURE: a list of markers. Each marker is one of: + - `skill-sequence: then within turns` + - `error-string: ""` + - `repeated-command: "" ≥ times` + - `repeated-file: read ≥ times` + - `compaction-then: ` + - `missed-trigger: for requests matching ""` + - `free: ` (use only the transcript to judge) + +Procedure: +1. `wc -lc` and the long-line check on CANDIDATE. Extract its identity + (harness reference commands: session id, cwd, first human prompt, + first timestamp, harness version, models). +2. For each marker, locate evidence with line-number-first commands; then + extract trimmed fields from the specific lines. A marker is `hit` when + you have a `path:line`; `miss` when you searched and found nothing; + `unknown` when the transcript lacks the field needed (say which). +3. Return exactly: + +``` +candidate: — +identity: , , "" +match: yes | partial | no +markers: +- : hit — : — "" +- : miss — checked +- : unknown — +``` + +`yes` = every marker hit; `partial` = at least one hit; `no` = none.