Repository navigation
fix(gh-aw): fail loudly when /squad parses no command (#1824) - #1832
Conversation
`/squad cast` silently no-opped with a green check unless the command started the issue body. `Parse Command` said "strip /squad prefix" — a position-0 assumption — and had no failure branch, so a run that cast nothing was byte-identical to one that worked. That is the first-run path for every new user. Parse Command is now three steps. PC-1 scans the whole body for the first /squad token and prints either the argument text or the NO_COMMAND sentinel. PC-2 routes NO_COMMAND to PC-3 and explicitly forbids falling back to `cast`. PC-3 is mandatory: it greps the body, emits ::error:: with the offending text quoted verbatim, comments on the issue, and exits non-zero. The parser is an LLM prompt, so the parse is declared as concrete shell commands in the markdown and the test extracts and executes those exact commands against real issue bodies — the declared contract and its observed behavior are independent sources. Two mutations were run against the fix. Reintroducing position-0 anchoring was caught by only 1 of 14 assertions: awk `^` anchors per record, so a line-anchored parser still handles a command on its own line after prose, and the headline "prose, blank line, then command" case passed under the reintroduced bug. Indented and mid-sentence cases were added to close that blind spot; the mutation then failed 2 tests naming the input. Replacing PC-3's diagnostic with a generic message — leaving every status signal intact, the shape that slipped through on #1793 — was caught by 3 assertions, because they require the diagnostic to contain the offending text rather than merely to exist. The suite resolves Git Bash rather than skipping on Windows, and asserts a shell was found, so it cannot report green for assertions that never ran. Closes #1824 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
🟡 Impact Analysis — PR #1832Risk tier: 🟡 MEDIUM 📊 Summary
🎯 Risk Factors
📦 Modules Affectedroot (1 file)
tests (1 file)
This report is generated automatically for every PR. See #733 for details. |
🛫 PR Readiness Check
PR Scope: 🔧 Infrastructure
|
| Status | Check | Details |
|---|---|---|
| ❌ | Single commit | 2 commits — consider squashing before review |
| ✅ | Not in draft | Ready for review |
| ✅ | Branch up to date | Up to date with dev |
| ❌ | Copilot review | No Copilot review yet — it may still be processing |
| ✅ | Changeset present | No source files changed — changeset not required |
| ✅ | Scope clean | No .squad/ or docs/proposals/ files |
| ✅ | No merge conflicts | No merge conflicts |
| ❌ | Copilot threads resolved | 2 unresolved Copilot thread(s) — fix and resolve before merging |
| ✅ | CI passing | All checks passing |
Files Changed (2 files, +445 −8)
| File | +/− |
|---|---|
test/gh-aw-command-parse.test.ts |
+280 −0 |
workflows/squad.md |
+165 −8 |
Total: +445 −8
This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved dispatch handling, shell-injection risk, and insufficient failure-path test coverage block approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Updates the Squad workflow to parse /squad commands throughout issue text and fail visibly when none is found.
Changes:
- Adds sentinel-based parsing and explicit failure routing.
- Adds shell-backed parser and diagnostic tests.
File summaries
| File | Review findings |
|---|---|
workflows/squad.md |
Moderate (3 votes): manual dispatch values bypass /squad parsing. Critical (1 vote): verbatim issue text is interpolated unsafely into shell commands. |
test/gh-aw-command-parse.test.ts |
Moderate (2 votes): PC-3 tests do not execute or validate the diagnostic emission and non-zero exit. |
Review details
Suppressed comments (3)
test/gh-aw-command-parse.test.ts:91
- The positive parser cases here all exercise slash-prefixed bodies; none models a
workflow_dispatchevent, where the authoritativecommandinput is bare (implementinsquad-implement-worker.md). There is therefore no test that feeds that raw input through the same resolution/normalization, so the current regression (every manual dispatch yieldingNO_COMMAND) remains green. Add a dispatch-input case and assert that it reaches the requested mode.
const cases: Array<{ name: string; body: string; expected: string }> = [
// The regression that was never broken — kept only as a floor.
{ name: 'command at position 0', body: '/squad cast', expected: 'cast' },
workflows/squad.md:244
- The
sub(/^.*\/squad/...)expression is greedy, so when a matching line contains multiple tokens (for example,Please run /squad cast, then /squad status) it strips through the last/squadand returnsstatus, despite the contract saying to take the first token. Extract the remainder from the firstmatch()result and add a multiple-token fixture.
printf '%s\n' "$SQUAD_TRIGGER_BODY" | awk '{sub(/\r$/,"")} !f && /(^|[[:space:]])\/squad([[:space:]]|$)/ {f=1; sub(/^.*\/squad/,""); sub(/^[[:space:]]+/,""); sub(/[[:space:]]+$/,""); print} END{if(!f) print "NO_COMMAND"}'
workflows/squad.md:280
- PC-3 always requires a comment on a triggering issue, but
workflow_dispatchalso reaches this step for a non-empty invalid command (for example,cst) without a triggering issue orissue_number; that input does not match the issue-bound activation guard. There is no issue target to comment on, so this path can fail before delivering the promised diagnostic/comment. Make the comment conditional on a resolved issue target and retain the annotation/non-zero failure for targetless manual runs.
3. Post one comment on the triggering issue reproducing that same text verbatim
and listing the valid commands from the Modes table.
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| it('PC-3 fails the run instead of reporting success', () => { | ||
| const pc3 = workflow.slice(workflow.indexOf(PC3_HEADING)); | ||
| expect(pc3).toContain('::error::'); | ||
| expect(pc3, 'PC-3 must fail the run').toMatch(/exit non-zero/i); | ||
| expect(pc3, 'PC-3 must not fall back to noop, which reports no comment').toMatch( | ||
| /Never call `noop`/i | ||
| ); |
| body chosen above) to `SQUAD_TRIGGER_BODY`, then run exactly this: | ||
|
|
||
| ```bash | ||
| printf '%s\n' "$SQUAD_TRIGGER_BODY" | awk '{sub(/\r$/,"")} !f && /(^|[[:space:]])\/squad([[:space:]]|$)/ {f=1; sub(/^.*\/squad/,""); sub(/^[[:space:]]+/,""); sub(/[[:space:]]+$/,""); print} END{if(!f) print "NO_COMMAND"}' |
| 1. Show the text actually present in the body: | ||
|
|
||
| ```bash | ||
| printf '%s\n' "$SQUAD_TRIGGER_BODY" | tr -d '\r' | grep -n -i -m 3 -F -- '/squad' || echo 'NO_SQUAD_TEXT_IN_BODY' | ||
| ``` | ||
|
|
||
| 2. Emit `echo "::error::Squad parsed no recognized command. Text seen: <verbatim | ||
| output of the command above>"`. Quote the observed text — never a generic | ||
| "unrecognized command" message with the offending input omitted. |
FIDO review — APPROVE WITH NITS(GitHub blocks a formal Approve on an own-account PR, so this lands as a comment. Verdict is in the text.) Adversarial re-derivation, not a read-through. I reproduced every claim independently in the worktree. Measured vs. assumed: Verified by measurement
Nits (non-blocking)
Residual I cannot close (stated plainly)PC-2/PC-3's "exit non-zero / never Ship it. Every load-bearing claim reproduced; the suite is mutation-sensitive on content (not status), fails red when it can't run, and is independent of the prose it guards. I tried to build a bypass and couldn't — and I'd rather say that than hand you a confident green. — FIDO |
FIDO — adjudicating the automated reviewer's findings (measured, not reasoned)My original review stands. Coordinator asked me to adjudicate the two findings my first pass didn't cover, plus confirm the convergence. Where I couldn't measure, I say so. Finding 1 — shell injection → NON-BLOCKING as stated; real hardening gapRan 8 hostile payloads through the exact PC-1 + PC-3 pipelines with the body via the environment (the channel the tests use):
Finding 2 —
|
| # | Finding | Call |
|---|---|---|
| 1 | Shell injection | NON-BLOCKING (pipeline provably safe) + required hardening on the assignment channel |
| 2 | Dispatch bare-command regression | BLOCKING — net regression on the autonomous relay |
| 3 | Greedy last-token | NON-BLOCKING, fix in-PR (same awk line) |
| 4 | PC-3 emission/exit | Inherent; record as known limit; diagnostic half already closed |
Overall: MERGE AFTER FIXES — rework the dispatch path. Finding 2 blocks. Finding 3 rides the same line. Finding 1 isn't a demonstrated exploit in the diff but leaves a security-critical step to LLM discretion — pin a safe channel. My #1824 assessment is unchanged: the issue-path fix is sound; the dispatch-path handling introduced alongside it is not.
— FIDO. Measured what I could; named what I couldn't.
FIDO's BLOCKING review on #1832: the /squad parser hardening was tested in isolation and broke its callers. workflow_dispatch delivers a BARE command -- squad-implement-worker.md relays {"command": "implement"}, and squad.md's own input schema documents cast/implement/connect org/repo, none carrying a /squad prefix. PC-1 returned NO_COMMAND for every one of those, routing a structurally valid run into PC-3 and hard-failing it. Manual dispatch and the autonomous implement relay worked before the hardening and did not after. Add Step PC-0, which normalizes the dispatch input before PC-1 sees it: trim, leave an existing /squad prefix alone (idempotent), otherwise prepend one, and emit EMPTY_DISPATCH for empty input so the activation guard halts it silently instead of PC-3 commenting and failing (PR #1777, junk issues #12/#14). PC-1 is deliberately NOT loosened. The paths are asymmetric: on the comment path a missing token IS the error condition, so loosening would reopen #1824. A regression test guards that wrong fix; mutation proves the guard fires. Also resolve the text contradiction that made the parser nondeterministic -- one passage said a dispatched value skips parsing, another marked PC-1 MANDATORY over it. Choosing a source now never skips parsing. Fix the greedy sub(/^.*\/squad/,"") that stripped through the LAST token on a line, resolving "/squad cast, then /squad status" to `status` -- a different mode than requested, silently. Use match() with substr($0, RSTART+RLENGTH). Record PC-3's "exit non-zero" as a known limitation: it is a prompt directive, not enforced code. Steps 1-3 are load-bearing because their output survives the run whatever exit status the agent chooses. Paste RETRO's shell input security contract as normative parser requirements (env: channel, four forbidden anti-patterns, six per-hop rules). Hop 1 -- compiler-channel compliance -- is UNMEASURED here: this repo ships no compiled gh-aw output. The verification gate is tracked in #1834 and explicitly not built in this PR; the text says so, so a declared requirement cannot be mistaken for an enforced one. Tests: 15 -> 27 in gh-aw-command-parse. Three mutations each turn the suite red AND name the offending input: PC-0 pass-through, greedy sub() restored, PC-1 loosened. All four gh-aw suites green, including strict gh aw compile and the prompt budget guard. Refs #1824, #1834 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
FIDO — Pass 3 re-review of rework
|
| # | Claim | Result |
|---|---|---|
| 1 | PC-0 fixes the BLOCKING dispatch regression (my Finding 2) | ✅ PC0("implement")→"/squad implement", idempotent, trims, ""→EMPTY_DISPATCH; all 6 dispatch modes resolve via PC1(PC0(x)). Resolved. |
| 2 | PC-1 deliberately NOT loosened; guard fires under mutation | ✅ mutation (c) fires the dedicated guard and reopens the #1824 headline |
| 3 | Greedy last-token fix (my Finding 3) | ✅ "Please /squad cast, then /squad status"→"cast, then /squad status" (first token wins) |
| 4 | Text contradiction resolved | ✅ :177 "skip the remaining sources" = source selection; :184-187 "Choosing a source never skips parsing … MUST be normalized by PC-0 before PC-1". PC-1 still MANDATORY. |
| 5 | Three mutations red-AND-naming | ✅ all three (below) |
| 6 | Tests 15 → 27 | ✅ 27 passed (27) |
| 7 | RETRO contract, hop-1 UNMEASURED, gate → #1834 | ✅ text says the gate is not implemented, "reviewed by hand, not enforced by CI"; cannot be misread as enforced |
| 8 | PC-3 exit-non-zero = known limitation (my Finding 4) | ✅ recorded as inherent to gh-aw |
Mutations (each red + names the input)
- (a) PC-0 pass-through → 7 red:
A workflow_dispatch of "implement" must resolve … expected 'NO_COMMAND' to be 'implement' - (b) greedy
sub(/^.*\/squad/,"")restored → 2 red:PC-1 must extract "cast, then /squad status" from "Please /squad cast, then /squad status" … expected 'status' - (c) PC-1 loosened → 9 red incl.
PC-1 must still reject "implement" … expected 'implement' to be 'NO_COMMAND'and /squad cast silently no-ops with a green check unless the command starts the issue body #1824 headlineWe should improve the docs.returned as prose
File restored via git checkout -- after each; final tree clean.
The two probes raised
- Probe A — PC-0
NR==1leading-newline: MEASURED, mechanism REAL, NON-BLOCKING (nit).PC0($'\nimplement')→"EMPTY_DISPATCH"(silent halt on a value carryingimplement);PC0($'implement\nfoo')→"/squad implement"(line 2 dropped); CRLF handled. Butcommandis aworkflow_dispatchstring input (squad.md:17-19); the only producers are the literal relaysquad-implement-worker.md:259 "command":"implement"and the single-line "Run workflow" UI — neither can carry a newline. Reachable only via a hand-craftedactions:writeAPI dispatch. Untested. Not a merge blocker, but a genuine latent silent-halt of this PR's own defect class — recommend a follow-up: pin it with a leading-newline test and/or scan the first non-empty line instead of bindingNR==1. - Probe B — enumerate every caller: MEASURED, clean. Grep of
workflows/*.mdfor"command":producers → exactly one (squad-implement-worker.md:259). No third bare-command dispatcher. The "hardened in isolation" bug class has no other instance.
CI
All green: Diff Size Guard, Architectural Review, Security Review, test (4m51s), Policy Gates, Changeset Drift. docs-quality/Scope Boundary skipping. MERGEABLE/CLEAN.
Verdict — APPROVE (merge as-is)
The pass-2 BLOCKING regression is fixed and regression-guarded by input-naming tests; Finding 3 fixed; the contradiction resolved; all mutations red-and-naming; the two inherent limits written down. One NON-BLOCKING nit: the PC-0 NR==1 leading-newline edge (Probe A) — latent, unreachable via known producers, untested; worth a follow-up test, not a blocker.
(Posted as COMMENTED — GitHub blocks a formal Approve from the PR-owning account.)
Closes #1835 PC-0 normalized the dispatched command with an awk program gated on NR==1, so a value whose first character is a newline put an EMPTY string on the scanned record: PC0($'\nimplement') returned EMPTY_DISPATCH. That routes to the activation guard, which halts with a ::warning:: and no comment by design (PR #1777, junk issues #12/#14) -- so a structurally valid implement dispatch would halt silently instead of running. Silent loss of a valid command is the same defect class #1824 and #1832 exist to close. Scan the first non-empty line instead: skip blank records, take the first one carrying content, and fall through to EMPTY_DISPATCH via END only when every record was empty or whitespace-only. Genuinely empty input still reaches the silent activation-guard halt, so #1777 stays closed. The one-liner is written without inter-statement spaces deliberately. workflows/squad.md sits ~37 bytes above the 5 KB floor of the 100 KB prompt budget asserted by gh-aw-quality's "reports combined bytes and headroom", so the readable spacing and an explanatory paragraph both overran the gate. The behavior is documented in the tests instead. That headroom is now ~65 bytes and is a blocker for the next change to this file; it needs its own issue. Tests: 27 -> 36 in gh-aw-command-parse. Reverting the awk to NR==1 turns four of the new cases red, and each failure message names the offending input, so a truncating parser cannot pass them. The whitespace-only cases ('', ' ', '\n', '\n\n', ' \n \t \n ') pin EMPTY_DISPATCH so the fix cannot overshoot, and the load-bearing PC-1 cases are untouched -- a bare token still fails on the comment path. Co-authored-by: brady gaster <bradygaster@github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…chives (#1838) * chore(squad): record #1793 session state, decisions and logs Merges both decision inbox records from the #1793/PR #1831 work (EECOM's CRLF working-tree repair, FIDO's independent review) and elevates the gate-instructions principle as its own entry: a correct check paired with a remediation hint that cannot observe the failure is still a broken gate. Also captures the gh-aw Tier 1/2/3 triage decisions and the .squad-must-not-be-gitignored decision, plus orchestration and session logs from both waves. Log files are force-added: .squad/ is hidden by the shared .git/info/exclude that #1826 re-creates, which makes new files under .squad/ silently unstageable while git status reads clean. Tracked files are unaffected. Refs #1793 #1826 #1830 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd * chore(squad): merge 3 gh-aw decision records, log #1824 wave Merges FIDO's #1832 review, RETRO's shell-input security contract, and Procedures' fail-loud command-parse record into decisions.md (48 -> 56 entries). Adds orchestration + session logs for the wave. Archival correctly skipped: all entries within a 4-day window, none older than the 7-day threshold. eecom/history.md held back pending repair of a dangling archive pointer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd * fix(squad): restore 3 archives lost by the 2026-08-20 summarization run A Scribe run at 2026-08-20T11:59:44-0700 wrote provenance pointing at history-archive-2026-08-20T11-59-44-0700.md into eecom, pao and procedures. None of the three files was ever created. The pointers dangled for two days with `git status` clean, because .squad/ was in .git/info/exclude (#1826) and new files there are silently unstageable. The trim landed in c508d86 -- titled "record gh-aw triage session state and repair archives". It performed the loss it claimed to repair. Recovered from the last committed pre-trim state (c508d86^ / 3dace32e). Not a proven byte-exact snapshot of disk at pointer-write time; uncommitted drift may have existed. Verified by blob SHA equality via `git ls-files -s`: eecom 771d9e0 15,063 B / 129 lines pao 012ea60 13,605 B / 120 lines procedures 1298528 13,653 B / 119 lines Content loss was invisible to size: procedures/history.md is 15,370 B today vs 13,653 B pre-trim -- 1,717 bytes LARGER -- and was still missing 35 non-blank lines. Later sessions appended while trimmed material stayed gone. Only content comparison detects this. pao was missing 44 lines. Also restored eecom/history.md provenance (H1 + 4 lines) stripped in an earlier repair attempt. Swept all 9 archive pointers across every agent: 0 remaining dangling. Refs #1836, #1826 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd * docs(squad): record the declared-vs-enforced principle and its corollary A requirement with no observer is documentation, not a rule. Six confirmations measured this working day, including the positive case -- RETRO's hop-1 shell contract, which openly labels itself unenforced and files the gate (#1834). That is the safe form; the failure mode is prose that reads as a guarantee. Sub-pattern: an unenforced requirement does not merely fail silently, it can emit an affirmative false claim of compliance -- "Roster set read from team.md" over a preset list; a commit titled "repair archives" that performed the loss (c508d86). False provenance is the worst case. Corollary -- a measured number that misstates its unit collapses the same way: the observation happened, but the number does not mean what it claims. Six instances on one file in one day, terminating in a false verdict: Scribe's "UNRECOVERABLE -- all commits < 10KB" against paths holding 41 commits at 13,605 B. A false measurement that becomes a conclusion to stop looking. Two rules: state the unit and the producing command for every number; prefer a measure nothing can reinterpret. During the repair, char count, line count and heading containment all passed on a BOM'd, newline-stripped file. Only the blob SHA could observe it. The record carries zero navigational line numbers -- every citation is a grep anchor that revalidates on read. Drift pressure scales with writer count, so decisions.md (union merge, every agent appending) is the last place to cite a line: the permanently-green restatement moved ~L604 -> L598 in one session, unannounced. Refs #1836, #1824, #1812, #1834 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd * chore(squad): merge 3 decision records, log the #1812 roster-binding wave Merges the gh-aw wave-1 decision inbox into decisions.md (56 -> 59 entries) and records the #1812 orchestration. Records merged: - procedures-1812-activate-roster-binding (Team Guard Step TG-2) - fido-1837-review (adversarial review, APPROVE WITH NITS) - flight-declared-vs-enforced (the governing principle) Archival was DISABLED for this run by coordinator override. decisions.md is 89,955 B, past the 51,200 aggressive gate, and that archival path has silently destroyed history twice in four days (repaired by #1782, then again in 6d8bb3f). A large decisions.md is a performance problem; a lossy archive is a correctness one. Tracked as #1836. Scribe reported this as "skipped by override" rather than "not required" -- it did not claim a measurement it never took. New files under .squad/log/ and .squad/orchestration-log/ required 'git add -f' (#1826). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd * fix(squad): repair BOM, CR, heading and link defects found in review GitHub's automated reviewer flagged four defect classes on PR #1838. All independently verified against the filesystem before repair. BOM (regression introduced by this PR): four history.md files carried a UTF-8 BOM (EF BB BF). parseHistory() preserves it but serializeHistory() matches titles with ^#\s+, so a BOM'd H1 fails to match and the title is silently dropped on round-trip. Char counts, line counts and heading containment all pass on a BOM'd file -- .NET ReadAllText strips BOMs silently, so only byte-level inspection observes it. Verified clean on origin/dev and BOM'd at HEAD, confirming this PR introduced them. Bare CR: eecom/history.md held one bare CR that had consumed the "r" of "research", rendering "esearch". HEAD bytes read "implement, " 0D "esearch" -- the CR replaced the letter rather than splitting around it, so restoring it required CR->r, not CR->space. Verified: 0 bare CR, 46 CRLF pairs preserved, 75 lines before and after, exactly one line differing. Adjacent H3 headings: three dated wrappers in decisions.md were each followed by an undemoted duplicate source heading, plus one ADDENDUM subsection using H3. Each caused the parser to flush a record with no body, producing four phantom decisions. The Scribe merge contract requires a spliced body's shallowest heading to land at H4; its parenthetical example shows only "## -> ####", and the rule was applied faithfully to H2 (measured: 0 stray H2) but never generalized to H3. Demoted to H4: 55 H3 / 85 H4, conserved, zero adjacent pairs remaining. Wrong repository: ten links pointed at bradygaster/friendly-guide -- a worktree name, not a repository -- across five log and orchestration-log files. Corrected to bradygaster/squad. Left untouched: the H3 inside a fenced block at decisions.md:986. The content is a normative example and is correct; the parser is the defect. Tracked separately as #1839. Refs #1838, #1839 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd --------- Co-authored-by: brady gaster <bradygaster@github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0e20e987-a753-41c4-a6f8-992d863a32cd
Working as Procedures (Prompt Architecture).
Closes #1824.
The defect
## Parse Commandinworkflows/squad.mdsaid "Strip/squadprefix, trim whitespace" — a position-0 assumption — and had no failure branch at all. Step 4, "Default tocastif empty", was the silent-success path. There was no state in which the router could report "I did not understand this," so a run that cast nothing and a run that worked were byte-identical to the user.The fix
Priority follows the issue's own framing: (2) fail loudly is the deliverable, (1) parse anywhere is today's instance.
/squadtoken and prints either the argument text or theNO_COMMANDsentinel.NO_COMMANDto PC-3 and explicitly forbids falling back tocast.::error::with the offending text quoted verbatim, comments on the issue, and exits non-zero. Nevernoop, never a success summary.How this is verified
The parser is an LLM prompt, not code, so there is no function to unit-test. The parse is instead declared as concrete shell commands in the markdown, and
test/gh-aw-command-parse.test.tsextracts those exact commands and executes them against real issue bodies. The declared contract and its observed behavior are independent sources — neither re-reads the other. This follows the existing TG-1 precedent intest/gh-aw-quality.test.ts.A test asserting "a command at position 0 parses" would guard the one case that was never broken. The load-bearing assertion is the inverse: a body that yields no mode must produce an explicit sentinel and a diagnostic naming the offending text.
Proved it can fail
workflows/squad.mdHonest caveat: that red is structural (
"### Step PC-1 ..." is missing), not behavioral. It proves coupling to the new contract, not sensitivity to a subtly-wrong parser. A behavioral pre-fix red is not constructible — there was no prior implementation to run, only prose. The mutations below are what actually tested sensitivity, and one of them found a real hole.Mutation 1 — reintroduce position-0 anchoring
Changed only the awk regex inside the fixed file; headings and prose untouched.
Only 1 of 14 assertions caught it. The headline case — prose, blank line, then the command, the literal #1824 scenario — passed under the reintroduced bug.
Cause:
^in awk anchors per record, not per body.printffeeds awk line-by-line, so a line-anchored parser still handles a command sitting on its own line after prose. That case cannot distinguish the two parsers.Added two cases whose only job is to carry that weight — an indented command and a mid-sentence command, both invisible to a line-anchored scan. Re-running the mutation then produced 2 failures naming the specific input. They look redundant next to the blank-line case; there is a comment in the test saying they are not.
Mutation 2 — generic diagnostic (the #1793 shape)
Replaced PC-3's grep with
echo 'unrecognized command'. Every status-shaped signal stayed intact:::error::still present, PC-2 still forbids the fallback, the run still fails. A status-only assertion passes this clean — exactly how a truncating parser slipped through on #1793.Caught by 3 assertions, because they require the diagnostic to contain the offending text, not merely to exist:
The gate cannot silently skip
HAS_POSIX_SHELL = existsSync('/bin/sh')is false on Windows, which silently skips 102 behavioral assertions ingh-aw-quality.test.ts— a permanently-green gate. This suite resolves Git Bash instead, and hard-asserts that a shell was found, so a suite that could not run reports red rather than green.Deliberate non-actions
/squadinside quoted lines and fenced blocks. Excluding them reintroduces a silent-skip path — the exact bug class PC-3 exists to remove. gh-aw'sslash_commandtrigger has already decided this is a squad command before the prompt sees the body. Documented in-file./bin/shskips were not fixed. Real, and worth its own issue; out of scope here and would blowDiff Size Guard.plan activate, downstream of routing. No shared code path withParse Command.Checks
npm run buildpasses. Build churn (twoSKILL.mdtemplates) was reverted, not committed;SKIP_BUILD_BUMP=1suppressed the version bumps.dev; the other 6 pass when run in a smaller batch (15s/9s/5s durations in the full run mark them as parallel-load timeouts).workflows/andtest/do not matchSDK_CLI_PATH_REGEX.