[bradygaster]

Closes #1822

What this is

#1822 is not "fix the broken query" — PR #1821 already did that, and did it well (exit-code capture, empty-cutoff guard, zero-rows scored INCONCLUSIVE, an explicit do not record false on the dispositive fields). #1822 asks for the mechanism that would have caught it, and would catch the next one.

This adds that mechanism.

The shape of the defect

E-scenario runbooks are executed by a human operator, never by CI, so a command that cannot run at all can sit in a runbook indefinitely. That is a documentation bug right up until it meets a gate whose pass condition is universally quantified — "every squad:{x} on a newly created issue must be a roster Name". Such a condition is vacuously true over the empty set.

So a broken command did not merely fail to inform the verdict. It wrote the passing answer into the two fields the schema marks # dispositive, where true is the FAIL evidence. An infrastructure failure minted the green result for the gate whose entire purpose was catching falsely-minted labels.

Measured against the bar in .squad/decisions.md (2026-08-20): "A test that passes while the system is broken is decoration, not a gate."

The gate

test/e2e-runbook-hygiene.test.ts lints every fenced command in .squad/e2e/ and fails the build on either property that made the silent pass possible:

1. gh invoked with --arg. It is a jq flag; gh has none. --jq consumes it as its expression and the rest degrade into stray positionals, so gh exits 1 with empty stdout rather than an obvious parse error — which is precisely why it survived review.
2. stderr suppressed without a nearby exit-code check. Suppression is fine for transcript readability; unpaired, it converts a hard failure into an empty result that reads as a pass.

Two details that matter:

• It joins shell line-continuations, so a flag is judged against the whole logical command. The original Phase 3b defect had --jq --arg on a continuation line — a per-line linter would have missed it.
• It reads only fenced blocks and skips comments. E4 documents this defect verbatim so the next reader understands why the guard exists; a linter that flagged its own postmortem would punish the documentation that prevents recurrence.

Verification

Mutation-tested, not just run. Reintroducing the original Phase 3b command — continuation and all — fails both rules with the offending file and line reported:

gh has no --arg flag. It exits 1 with empty stdout, which a
universally-quantified gate reads as a pass.
  .squad/e2e/ZZ-mutant-probe.md:6 — $rows = gh issue list ... --jq --arg t "$E4_START" ...

The real runbooks stay green throughout, including E4's prose description of the same bug.

Measured at introduction: 16 of 17 existing suppression sites already paired with an exit-code check. The single apparent miss was the comment describing the defect. So this codifies discipline the runbooks already practice rather than imposing a new one — it fails only on regression.

test/e2e-runbook-hygiene.test.ts + test/docs-links.test.ts: 6 passed.

The half that cannot be mechanized

Detecting "this gate is universally quantified" reliably from prose would produce false positives, so it is recorded as written convention in .squad/e2e/README.md instead:

• A universally-quantified gate must assert its input set is non-empty before evaluating. Zero rows is a distinct outcome from "all rows satisfied the predicate."
• Never let an infrastructure failure reach a # dispositive field. Non-zero exit or unset cutoff means INCONCLUSIVE, never a verdict value.

Cost

Executes no commands. Needs no fixture, network, or Actions minutes.

Note for the E4 re-run

Because #1821 is merged, E4 Phase 3b is already sound — the re-run is not blocked on this PR. What does block it is #1851: E1 and E4 both hardcode bradygaster/aspiregregator-squad-e2e, which no longer exists.

Owners

FIDO (test/CI gate) with Sims (runbook convention), per the issue.

No changeset — no packages/*/src/ changes.

[github-actions]

🟡 Impact Analysis — PR #1852

Risk tier: 🟡 MEDIUM

📊 Summary

Metric	Count
Files changed	2
Files added	1
Files modified	1
Files deleted	0
Modules touched	2

🎯 Risk Factors

• 2 files changed (≤5 → LOW)
• 2 modules touched (2-4 → MEDIUM)

📦 Modules Affected

squad-state (1 file)

• .squad/e2e/README.md

tests (1 file)

• test/e2e-runbook-hygiene.test.ts

---

This report is generated automatically for every PR. See #733 for details.

[github-actions]

⚠️ Squad File Leakage Detected

The following .squad/ files were modified in this PR:

• .squad/e2e/README.md

These files affect team routing, agent charters, and decisions.
If intentional, ensure approval from the team lead.

[github-actions]

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit 14ccabb

PR Scope: 🔧 Infrastructure

⚠️ 2 item(s) to address before review

Status	Check	Details
✅	Single commit	1 commit — clean history
✅	Not in draft	Ready for review
✅	Branch up to date	Up to date with dev
❌	Copilot review	No Copilot review yet — it may still be processing
✅	Changeset present	No source files changed — changeset not required
✅	Scope clean	⚠️ PR includes 1 .squad/ file(s) — ensure these are intentional
✅	No merge conflicts	No merge conflicts
❌	Copilot threads resolved	1 unresolved Copilot thread(s) — fix and resolve before merging
✅	CI passing	All checks passing

Files Changed (2 files, +213 −0)

File	+/−
.squad/e2e/README.md	+16 −0
test/e2e-runbook-hygiene.test.ts	+197 −0

Total: +213 −0

---

This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

[Copilot]

🟡 Changes recommended

The new hygiene test has an off-by-one bug vs its own documented window semantics, which can cause incorrect lint failures.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a CI gate to lint .squad/e2e/ runbook fenced-command blocks for failure-masking patterns that can produce vacuous “pass” outcomes (notably gh --arg misuse and stderr suppression without an exit-code check), and documents the underlying runbook conventions in the E2E README.

Changes:

• Introduces test/e2e-runbook-hygiene.test.ts to statically lint fenced runbook commands (including joining line continuations) for two specific regression shapes.
• Documents the non-mechanizable “universally quantified gates must assert non-empty input” convention in .squad/e2e/README.md.

File summaries

File	Description
test/e2e-runbook-hygiene.test.ts	New Vitest suite that lints fenced runbook command blocks for gh --arg and unpaired stderr suppression patterns.
.squad/e2e/README.md	Adds a “runbook conventions” section explaining the defect class and pointing to the new hygiene gate.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.