Repository navigation
Certify roster set via emitting Team Guard Step TG-2 (Closes #1812) - #1837
Conversation
Activate bound `squad:{agent}` labels to the hardcoded preset roster
(lead/reviewer/devrel/security/docs) instead of the repo's committed cast,
and printed a false provenance sentence claiming it read team.md. Prose
hardening had failed repeatedly.
Add Team Guard Step TG-2: a bash block (modeled on the TG-1 TEAM_PRESENT
guard) that reads committed-HEAD team.md, finds the `## Members` `Name`
column by header, and emits `ROSTER_MEMBER: {name}` per row (lowercased) or
`ROSTER_UNREADABLE: {reason}`. Every minting/binding site (accept, activate
gate, plan, impl, Check 10) binds only to TG-2 stdout, so the provenance
claim is true by construction. Unreadable rosters halt with the named
reason -- never a preset fallback, never a provenance sentence for a read
that did not happen.
Add test/gh-aw-activate-roster-binding.test.ts: extracts the TG-2 block and
runs it against real committed-HEAD git repos (10 tests). Mutation-proved
M1-M7 (each reddens naming the offending input); M8 (CR-strip) could not be
reddened on Windows git-bash and is documented as a named boundary.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
🟡 Impact Analysis — PR #1837Risk tier: 🟡 MEDIUM 📊 Summary
🎯 Risk Factors
📦 Modules Affectedroot (1 file)
squad-state (1 file)
tests (1 file)
This report is generated automatically for every PR. See #733 for details. |
🛫 PR Readiness Check
PR Scope: 🔧 Infrastructure
|
| Status | Check | Details |
|---|---|---|
| ❌ | Single commit | 2 commits — consider squashing before review |
| ✅ | Not in draft | Ready for review |
| ✅ | Branch up to date | Up to date with dev |
| ❌ | Copilot review | No Copilot review yet — it may still be processing |
| ✅ | Changeset present | No source files changed — changeset not required |
| ✅ | Scope clean | |
| ✅ | No merge conflicts | No merge conflicts |
| ❌ | Copilot threads resolved | 5 unresolved Copilot thread(s) — fix and resolve before merging |
| ✅ | CI passing | All checks passing |
Files Changed (3 files, +532 −62)
| File | +/− |
|---|---|
.squad/decisions/inbox/procedures-1812-activate-roster-binding.md |
+146 −0 |
test/gh-aw-activate-roster-binding.test.ts |
+305 −0 |
workflows/squad.md |
+81 −62 |
Total: +532 −62
This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.
|
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings remain in TG-2 enforcement, parsing, roster matching, and label handling.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds TG-2 roster certification from committed .squad/team.md to prevent preset roster leakage during planning and activation.
Changes:
- Adds committed-HEAD roster parsing and validation.
- Rewires roster binding guidance.
- Adds behavioral tests and a decision record.
File summaries
| File | Description |
|---|---|
workflows/squad.md |
Adds TG-2 and updates roster-binding rules. |
test/gh-aw-activate-roster-binding.test.ts |
Tests roster extraction and edge cases. |
.squad/decisions/inbox/procedures-1812-activate-roster-binding.md |
Documents the design and limitations. |
Review details
Suppressed comments (9)
test/gh-aw-activate-roster-binding.test.ts:29
- The workflow text is read without normalizing CRLF, and only the fence regex is CRLF-aware; the extracted shell block therefore retains
\rat the end of each command. On the Windows path this suite explicitly supports, Git Bash can parse tokens such asthen\rincorrectly and fail before the behavioral cases execute. Normalize the workflow text to LF as the other gh-aw suites do.
const workflow = readFileSync(SQUAD_WORKFLOW, 'utf8');
test/gh-aw-activate-roster-binding.test.ts:186
- These tests execute only the TG-2 producer. They do not assert that the five downstream binders (plan, accept, implementation, Check 10, and activate) actually consume TG-2 output, so reverting any one of those sites to an independent or preset roster would leave all ten cases green. Add structural assertions enumerating those sections and requiring the TG-2 certified-set contract, even though the applied-label subset is unavailable cross-job.
expect(TG2_BLOCK.length, 'TG-2 bash block extracted empty from workflows/squad.md').toBeGreaterThan(0);
expect(TG2_BLOCK, 'TG-2 block must emit ROSTER_MEMBER: lines').toContain('ROSTER_MEMBER:');
expect(TG2_BLOCK, 'TG-2 block must emit ROSTER_UNREADABLE: on failure').toContain('ROSTER_UNREADABLE:');
// It must read the *committed* revision, not the working tree — the entire point.
expect(TG2_BLOCK, 'TG-2 must read the committed HEAD revision of team.md').toMatch(/git show HEAD:\.squad\/team\.md/);
workflows/squad.md:1319
- TG-2's stdout is not connected to the
create-issuepayload or to the executor:create-issuestill accepts a model-generated arbitrary label list. These lines are therefore prompt-only; a model can still emitsquad:leador omit all roster labels while the TG-2 tests pass. An enforceable check in the label-application path (or a safe-output/schema constraint) is needed before this guarantees the #1812 binding invariant.
4. For every `Agent` value, mint `squad:{agent}` only when its lowercased form matches
a certified `ROSTER_MEMBER:` name, or the value is exactly `@copilot`.
5. A value matching no certified name and not `@copilot` MUST NOT become a
`squad:{agent}` label: apply only `squad` for that issue and record the value under a
`Non-roster agent values` heading, naming the certified set it should come from.
workflows/squad.md:473
- Valid Markdown tables may omit the final pipe. For a separator such as
|------|------, this regex does not match, so the separator's second cell is emitted asROSTER_MEMBER: ------before the real names, creating a bogus roster label. Allow the final delimiter to be optional and add a regression case.
if($0 ~ /^\|[-: |]*\|$/)next
workflows/squad.md:477
__NOCOL__is an in-band marker and is detected with substring matching. A legitimate cast name containing that token (including the exact name) makes a readable table reportROSTER_UNREADABLE, dropping valid bindings. Use awk's exit status or a separately scoped status channel instead of mixing a marker into member data.
if printf '%s\n' "$ROSTER" | grep -q "__NOCOL__"; then
workflows/squad.md:868
- This step still tells the model to read the working-tree
.squad/team.mddirectly, contradicting the invariant above that binders use only TG-2 stdout. In the targeted failure mode that copy can be the uncommitted preset while TG-2 intentionally reads committed HEAD, so the planner can repopulateOwner/Agentfrom the wrong roster. Remove the independent roster read and use only the certified output for binding.
3. Read `.squad/team.md` if it exists. **Owner/Agent binding rule:** permitted
`Owner`/`Agent` values are Team Guard Step TG-2's certified roster set — the
`Name` column of `## Members` in **this repository's** `.squad/team.md` — plus
`@copilot`. Map each item's domain to a member via `.squad/routing.md` and emit
that member's exact `Name` cell — never a recalled name, never another column
workflows/squad.md:1316
- This condition permits activation to mint
squad:@copilot, but TG-2 explicitly says that@copilotmust never become asquad:{name}label; the accept path also applies onlysquadfor that fallback. In addition, the check is case-insensitive while{agent}is left raw, so an exact Name such asKeatoncan producesquad:Keatoninstead of the lowercased label. Make the label token the lowercased certified name and exclude@copilotfrom agent-label creation.
4. For every `Agent` value, mint `squad:{agent}` only when its lowercased form matches
a certified `ROSTER_MEMBER:` name, or the value is exactly `@copilot`.
workflows/squad.md:1125
- The structural validation immediately above still defines agent validity by reading
.squad/team.mddirectly (line 1117), not by checking TG-2'sROSTER_MEMBER:output. A working-tree/default-preset value can therefore pass this validation even when it is absent from the certified stdout and be posted in the implementation artifact. Update this validation to use only TG-2's emitted set, matching the new binder.
Re-check every `Agent` against the Step 2 binding rule before posting.
workflows/squad.md:462
- A committed empty (or newline-only)
team.mdmakesgit showsucceed but leavesTEAM_MDempty, so this branch reports.squad/team.md absent from HEAD, which is false and hides the distinct empty/no-section failure. Capture thegit showexit status separately from the content check and report the actual condition.
TEAM_MD="$(git show HEAD:.squad/team.md 2>/dev/null)"
if [ -z "$TEAM_MD" ]; then
- Files reviewed: 3/3 changed files
- Comments generated: 5
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if(col==0){for(i=1;i<=NF;i++){h=$i;gsub(/^[ \t]+|[ \t]+$/,"",h);if(h=="Name")col=i}next} | ||
| if($0 ~ /^\|[-: |]*\|$/)next | ||
| if(col==0)next | ||
| n=$col;gsub(/^[ \t]+|[ \t]+$/,"",n);if(n!="")print tolower(n)} |
| 6. Completeness: when the plan names at least one roster `Agent`, at least one | ||
| `squad:{agent}` label MUST be applied across the created issues. Zero labels on a | ||
| plan with roster owners is a binding failure, not a pass — report it, don't proceed | ||
| silently. |
| Every step that mints a `squad:{name}` label or binds an `Owner`/`Agent` value binds **only** to this command's stdout. Run once. | ||
|
|
||
| ```bash |
| elif ! printf '%s\n' "$TEAM_MD" | awk '{sub(/\r$/,"")} /^## Members/{f=1} END{exit !f}'; then | ||
| echo "ROSTER_UNREADABLE: no ## Members section in .squad/team.md" | ||
| else | ||
| ROSTER="$(printf '%s\n' "$TEAM_MD" | awk -F'|' ' | ||
| {sub(/\r$/,"")} | ||
| /^## Members/{f=1;next} |
| /^## Members/{f=1;next} | ||
| f&&/^#/{f=0} | ||
| f&&/^\|/{ | ||
| if(col==0){for(i=1;i<=NF;i++){h=$i;gsub(/^[ \t]+|[ \t]+$/,"",h);if(h=="Name")col=i}next} |
FIDO adversarial review — VERDICT: APPROVE WITH NITSMeasured in a branch worktree with Standing asks — all METPer-case mutations (7), all red + naming input:
Caller enumeration (incl. outside Honesty claims — both VERIFIED by measurement
#1784 completeness rule — CONFIRMEDActivation step 6: "Zero labels on a plan with roster owners is a binding failure, not a pass." Prevents "refused everything" masquerading as "bound correctly" — the exact #1784 false pass. Prose compression — nothing load-bearing dropped(Reconciled against The finding (main ask) — L1117: REAL, NON-BLOCKINGThree validation sites; two rewired, one missed:
Non-blocking because the authoritative minting site and the authoritative gate (Check 10) are both TG-2-bound; L1117 is an inline sanity pre-check re-validating values Step 2 already sourced from TG-2, and any miss is caught by TG-2-bound Check 10 downstream. But a real should-fix — it reintroduces the untrusted source at a validation gate, the exact residual that let #1812 recur 5×. One-line fix: rebind L1117 to "TG-2's certified roster set ( CI:
|
FIDO nit on #1837: Step 3 "Validate Structure" (L1117) still derived agent validity from the untrusted file (.squad/team.md) while the binder (L1111) and Check 10 (L1165) bind to Team Guard Step TG-2's certified ROSTER_MEMBER: set. A validator that derives membership differently from the binder can pass work the binder would reject. Rewire the validator to the certified set; retain the verbatim `Name`-column phrase as provenance. Extends #1793: the criterion a check applies is part of the check. Closes #1812 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9cace6b6-8abc-4aa4-ac36-db3536dbb3bb
Closes #1812
Working as Procedures (Prompt Engineer).
The defect (measured)
On fixture
aspiregregator-squad-e2e(run32471509974),plan activateprinted a provenance sentence claiming it read the roster from.squad/team.md## Members->Namecolumn, reportinglead, reviewer, devrel, security, docs. The fixture's real cast isKeaton, McManus, Fenster, Hockney, Kint. The five reported names are exactly thesquad init --preset defaultscaffold. Two defects: (1) activate bound the hardcoded preset roster, not the committed cast; (2) it claimed a provenance it did not have. Downstream, activate matched the planner's real names against the hardcoded list, found none, and applied nosquad:{agent}label — so #1784 Condition 2 "passed" only because activate refused everything. Refusal and correct binding are indistinguishable from the outside.The fix (structural, not prose)
Add Team Guard Step TG-2 — Certify the Roster Set, modeled on the existing TG-1
TEAM_PRESENT/TEAM_ABSENTguard. It:.squad/team.md(git show HEAD:— working-tree preset scaffolds are invisible),## MembersNamecolumn by header (not position — a| Role | Name |table would otherwise leak the Role column with a true-looking citation),ROSTER_MEMBER: {name}per data row, or a singleROSTER_UNREADABLE: {reason}.Every minting/binding site (accept
squad:{owner}, activate label pre-flightsquad:{agent}, plan/impl binders, Check 10) binds only to TG-2's stdout. The summary can only reproduce names the command produced, so provenance is true by construction.ROSTER_UNREADABLE:halts binding with the named reason — never a preset fallback, never a provenance sentence for a read that did not happen.Prose was chosen against deliberately: the file was already hardened with bold, repeated roster prose and the defect happened anyway. TG-2's stdout is an observable artifact a test can assert; a directive's compliance cannot be observed.
What I measured
test/gh-aw-activate-roster-binding.test.ts(10 tests) extracts the TG-2 bash block and runs it against real committed-HEAD git repos. All green. Full gh-aw family: 172 passed / 13 skipped across 5 suites.tolower-> lowercase cases red, namingROSTER_MEMBER: Keaton.ROSTER_MEMBER: lead/reviewer(the exact plan activate reads a hardcoded roster, not .squad/team.md — and reports a false provenance (re-diagnoses #1784) #1812 anti-pattern).__NOCOL__sentinel -> no-Name-column case red (misdiagnosed reason).git show HEAD:->cat-> preset-leak case red, naming the working-treeROSTER_MEMBER: lead/reviewer/devrel/security/docs; also reddens the staticgit show HEADguard.ROSTER_UNREADABLEwith a spuriousROSTER_MEMBER: ghost-> each respective case red namingghost.squad:{...}and roster site acrossworkflows/grepped. All minting/binding sites bind to TG-2. The two other roster consumers —squad-implement-worker.md(routes by an already-certified label) andshared/squad.md(gh-aw lifecycle overwrites a merged cast with the default preset on later runs #1657squad initexistence guard) — derive no roster and do not share the defect. No compiled.lock.ymlartifacts are committed, so nothing needs regeneration.test/gh-aw-*; all pinned substrings survive (verified by the green plan-lifecycle + quality suites). No load-bearing prose dropped.What I could NOT measure (named boundaries)
sub(/\r$/,"")could not be reddened on the Windows git-bash test host —git showemits LF there (CR normalized before awk), and a trailing pipe quarantines any CR into an unread field. The strip is retained as defense-in-depth (load-bearing on Linux runners for irregular tables); its behavior is reasoned for Linux, not measured on Windows. The CRLF test was rewritten to assert the end-to-end parse invariant (reddenable by M1/M4) rather than claim to prove the strip.applied_labels ⊆ emitted_roster_members: not reachable under gh-aw safe-outputs — labels are applied by a separate executor job, and TG-2's stdout is not promotable to a cross-job artifact. Emission is shell-enforced/assertable; consumption stays a prompt directive (the good kind of unenforced — it labels itself). The≥1 label appliedcompleteness rule remains the guard that stops "refused everything" from masquerading as "bound correctly".Files
workflows/squad.md— TG-2 block + rewired binders (net +80/-61; byte budget passes, 87B margin above floor).test/gh-aw-activate-roster-binding.test.ts— new suite (10 tests)..squad/decisions/inbox/procedures-1812-activate-roster-binding.md— decision record (Scribe merges).No
packages/*/src/touched — no changeset required.