Skip to content

Harden published Squad GH-AW setup from one clean end-to-end run #1903

Closed

Description

@bradygaster

Summary

A clean consumer-repository run reproduced several documentation gaps in the published Squad GH-AW setup path. The installation and implementation worked, but the guide should explicitly require strict compilation before the bootstrap PR and accurately describe first-install approval, generated files, standalone activation, and recovery signals.

This issue tracks only findings from this single end-to-end run. It does not propose backend repository setup automation, new product features, or multi-repository orchestration.

Run evidence

  • Consumer repository: Private consumer repository (Run 1)
  • Bootstrap PR: Private Run 1 bootstrap PR Feature: Issue Assignment & Auto-Triage for Squad Members #4
  • Template: a private Aspire template repository
  • Generated name followed AspireApp_{descriptive-slug}_{yyyyMMddTHHmmssZ}
  • gh-aw: v0.86.2
  • Squad standalone default: v0.13.1

Exact client-run steps

  1. Created the private consumer repository from a private Aspire template repository.
  2. Enabled Actions-created pull requests while keeping default workflow permissions read-only.
  3. Built the template before Squad installation: .NET build/test and frontend lint/build completed with pre-existing warnings only.
  4. Created chore/squad-gh-aw-bootstrap.
  5. Ran gh aw add for:
    • squad.md@dev
    • squad-implement-worker.md@dev
    • squad-deps-worker.md@dev
    • squad-review.md@dev
  6. Reviewed the first-install safe-update report and ran gh aw compile --strict --approve for the documented restricted secrets and Squad init action.
  7. Verified all four authored .md files and four generated .lock.yml files existed.
  8. Committed only .gitattributes, .github/aw/, .github/skills/, and .github/workflows/; pushed and opened the bootstrap PR.
  9. Reran the exact command gh aw compile --strict in the consumer repository. All four workflows succeeded with no tracked-file drift.
  10. Requested @copilot review and waited for CI; reported checks passed.

Confirmed documentation problems

  1. Strict compile is omitted from the supported bootstrap path. The published quick start relies on gh aw add compilation and only mentions non-strict gh aw compile as troubleshooting. The supported path must require gh aw compile --strict in the consumer repository before committing or opening the bootstrap PR.
  2. The documented flow pushes directly instead of opening a human-reviewable bootstrap PR. The client-owned boundary requires a branch, strict compile, generated-diff review, and PR.
  3. The installed workflow list omits the dependency worker pair. The complete top-level set is four .md/.lock.yml pairs.
  4. First-install approval and final validation are conflated. gh aw add reported the expected restricted secrets (SQUAD_GITHUB_APP_PRIVATE_KEY, SQUAD_GITHUB_TOKEN) and bradygaster/squad/.github/actions/squad-init. Approval should be documented as conditional and reviewed; a final unqualified gh aw compile --strict should still be the validation gate.
  5. The activation description is stale. The published “How the workflow runs” section still says activation installs @bradygaster/squad-cli and broadly says state does not persist. v0.13.1 downloads the standalone release without npm, preserves committed cast state, validates no-npx wiring, runs health, and uploads state only on success.
  6. Optional editor output is unexplained. gh aw add left .vscode/settings.json untracked, which caused gh pr create to warn about one uncommitted change. The guide should state that this file is optional and excluded from the documented staging set unless the team wants to share it.
  7. The known strict-compile warning lacks context. squad.md warns that both slash-command and github-actions[bot] triggers are configured. This is intentional for controlled worker continuation dispatches, but users need to know that this single warning is expected and additional warnings are not.
  8. Recovery guidance is incomplete. The guide should state the smallest recovery action for activation failure, interrupted commands, existing Cast/implementation PRs, and unchanged reviewer heads.

Expected result

The published guide provides one copy/paste-accurate client-owned sequence:

  1. create a bootstrap branch;
  2. run the four-workflow gh aw add command;
  3. review and conditionally approve only the documented first-install safe updates;
  4. run gh aw compile --strict in the consumer repository;
  5. verify all eight source/lock files;
  6. stage only documented generated surfaces;
  7. inspect the staged diff and confirm no deletions;
  8. open a bootstrap PR, request Copilot review, wait for checks, and require human approval;
  9. describe npm-free v0.13.1 activation and committed-state persistence accurately;
  10. document rerun and interruption recovery without duplicate output.

Acceptance criteria

  • Published quick start includes gh aw compile --strict before the bootstrap PR.
  • First-install --approve is conditional, bounded to the documented secrets/action, and separate from final strict validation.
  • The guide lists all four workflow source/lock pairs.
  • The guide stages the observed generated surfaces and explains optional .vscode/settings.json.
  • The guide opens a human-reviewable bootstrap PR rather than pushing setup directly to the default branch.
  • Standalone v0.13.1 activation, init, health, state upload, and committed-state behavior are described accurately.
  • The intentional bot-trigger warning and stop conditions for other warnings/errors are documented.
  • Rerun and interrupted-run recovery steps are explicit and idempotent.
  • Repeating this consumer bootstrap from the updated guide produces a complete PR with all workflows strictly compiled and no ambiguous step.

Activity

  1. bradygaster commented on Aug 26, 2026

    @bradygaster
    CollaboratorAuthor

    Test-run issue content must be explicit

    For each consumer repository, the runbook must distinguish two GitHub issues and specify their exact purpose and minimum content:

    Casting issue

    • Purpose: create and review the repository's Squad only.
    • Title identifies the run as Squad casting validation.
    • Body describes the repository's existing languages, frameworks, CI, testing surfaces, and the desired team composition.
    • Body explicitly says not to implement an application feature.
    • Trigger is exactly /squad cast.
    • Evidence records the issue, Cast run, Cast PR, and the three persisted casting files.

    Intent issue

    • Purpose: drive one explicit test intent through research, planning, activation, and readiness for implementation.
    • Title names the concrete vertical slice or bounded change under evaluation.
    • Body includes scope, constraints, acceptance criteria, API/UI/test expectations, and explicit exclusions.
    • The intent issue is separate from the casting issue; casting context must not be reused as an ambiguous implementation request.
    • Evidence records research, plan, activation, generated issues, ownership labels, and dependency links.

    Current two-run stopping point

    For both independent repositories, exercise: bootstrap → cast → intent → research → plan → activate → implementation ready. Stop before /squad implement executes. Record friction and opportunities first; do not make broad workflow changes between runs.

    A shorter research/plan/activate path is a P3 optimization opportunity, not part of the current validation gate.

  2. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 1 — status verification finding

    After merging consumer Cast PR #6, /squad status completed successfully in run 33025670052 and reported the expected six-member active roster. It also reported:

    .squad/casting/registry.json still reflects the stale preset-default scaffold ... and hasn't been updated to match the current cast names in team.md.

    This is a concrete Cast-state consistency defect in the same end-to-end run: the generated Cast PR included and merged .squad/casting/registry.json, but the subsequent status check considered it stale. The workflow currently labels the mismatch cosmetic, yet registry/roster consistency should be verified because later routing and recasting rely on committed Cast state.

  3. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 1 — research roster binding defect

    /squad research completed successfully in consumer run 33026053482, but its R10 finding contradicts the committed and status-verified roster. It claimed .squad/team.md has only generic lead/reviewer/devrel/security/docs roles and recommended assigning implementation to lead. The active roster is Keaton, McManus, Fenster, Hockney, Verbal, and Kujan, including dedicated backend, frontend, data, DevOps, and test specialists.

    This is actionable because research explicitly feeds downstream ownership recommendations; a stale or incorrectly read roster can produce invalid task owners/labels during plan activation. Continue the run to determine whether /squad plan independently reloads the correct roster or propagates the mismatch.

  4. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 1 — planning propagated stale owners

    /squad plan completed successfully in consumer run 33026481037 but propagated the research roster defect into the implementation plan. It assigned tasks to lead, devrel, and reviewer and stated the roster has no backend/frontend split. None of those are active cast member names in the merged roster; correct domain owners include McManus, Fenster, Verbal, and Kujan.

    This is now beyond cosmetic status output: the plan's Owner cells drive squad:{owner} labels and downstream implementation dispatch. Continue with the documented /squad plan accept fast path to observe whether the activation roster-binding gate rejects or repairs these stale owners, or incorrectly creates generic labels.

  5. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 1 — activation reached implementation-ready with degraded ownership

    /squad plan accept completed successfully in run 33026843469 and created epic #8, phase #9, and task issues #10–#15. The workflow stopped at the requested implementation-ready boundary; /squad implement was not invoked.

    Activation did not repair the stale lead/reviewer/devrel owners. It emitted an explicit warning that those values do not match the certified roster and therefore minted no squad:{owner} labels. Every created issue has only the generic squad label. The backlog exists, but it is not roster-routable without manual reassignment, so the end-to-end experience is degraded at the handoff to implementation.

    Run evidence:

  6. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 1 — acceptance idempotency passed

    Reran /squad plan accept on consumer issue #7 in run 33027259229. The workflow recognized the prior activation, referenced the existing epic/phase/tasks (#8–#15), and created no new issues. Open/all issue count remained 10 with maximum issue number 15.

    This portion of the experience is working: duplicate acceptance is safe and provides a clear next action without duplicating backlog items.

  7. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 2 — bootstrap reproduced documentation gaps

    Fresh repository: the private Run 2 consumer repository; bootstrap PR #3 merged after green CI and Copilot review.

    Following the published four-entry gh aw add ...@dev command produced four source/lock pairs, including squad-deps-worker.md + .lock.yml. The published "installed top-level workflow set" lists only three pairs and omits this generated dependency worker pair.

    The install also generated .github/aw/actions-lock.json, but the published commit command stages only .gitattributes, .github/workflows/, and .github/skills; it omits .github/aw/actions-lock.json. Run 2 included that generated action lock deliberately. The docs should explicitly stage it.

    gh aw compile --strict --approve followed by exact gh aw compile --strict succeeded for all four workflows; post-commit strict compilation produced no tracked drift.

  8. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 2 — cast completion navigation friction

    /squad cast completed successfully in run 33028371031 and opened draft PR #5. The user-facing completion did not link or name the created PR; it said PR: opening now (see the Pull Requests tab...) even though #5 had been created. The local gh run watch --exit-status also returned exit code 1 while the final run state and all jobs were success, so direct run-state verification was required.

    The primary UX defect is the missing concrete PR reference: Cast completion should report the verified PR number/URL rather than asking the user to search the PR tab.

  9. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 2 — workflow-created Cast PR required manual CI approval

    Cast PR #5 was created successfully, but its application CI run 33028789813 ended as action_required. The PR was authored from the workflow-generated branch and required an explicit maintainer approval (POST .../actions/runs/33028789813/approve) before the template's .NET/frontend CI could execute.

    This is a concrete clean-repository UX gate not surfaced by the published Cast instructions or completion response. Users following the documented "review, merge, then status" path may see an apparently blocked PR without being told to approve its workflow run.

  10. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 2 — stale planning owners reproduced independently

    Run 2 /squad status correctly recognized the active roster (Keaton, Fenster, Hockney, Kint, McManus), and research run 33029698321 did not claim a stale roster. Nevertheless, /squad plan run 33030049219 assigned all four tasks to generic lead or reviewer instead of the certified cast.

    This reproduces the implementation-handoff defect independently of run 1's incorrect status/research wording. The fast-path planner is not binding task owners from the active cast even when team status is correct.

  11. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Two-run consumer validation complete

    Both clean repositories reached implementation-ready state. No /squad implement command was issued.

    Run 1

    Run 2

    Reproduced findings

    1. Planning does not bind active cast owners. Both plans used generic lead/reviewer/devrel values instead of roster names. Both activations therefore created only generic squad labels and required manual assignment before specialist routing. This is the highest-priority implementation-handoff defect.
    2. Published setup omits generated artifacts/validation. The documented workflow-set list omits the generated dependency worker pair; the commit command omits .github/aw/actions-lock.json; strict compilation should be explicit.
    3. Workflow-created Cast PRs can require manual CI approval. Run 2 application CI was action_required until a maintainer approved run 33028789813; the current instructions/completion response do not surface this gate.
    4. Cast completion can omit its verified PR reference. Run 2 said “PR: opening now” and told the user to search the Pull Requests tab instead of linking feat: Issue Assignment & Auto-Triage for Squad Members #5.
    5. Roster-state interpretation is inconsistent. Run 1 status/research described stale generic state, while run 2 status correctly recognized its cast. Planning still used generic owners in both cases.

    Working behavior

    • Standalone v0.13.1 activation succeeded without npm.
    • Four workflows installed and compiled strictly.
    • Bootstrap and Cast PR review/CI paths completed.
    • Research and planning produced usable content.
    • Dependencies were preserved in generated task bodies.
    • Repeated plan acceptance was a clear no-op in both runs.

    The P3 opportunity remains a shorter, consistently explained research → plan → activate journey. No workflow or documentation fixes were applied during these runs; this comment closes the evidence-gathering phase before changes are selected.

  12. added 8 commits that reference this issue on Aug 27, 2026
    06bed29
    bdc383e
    a05e33a
    054181e
    d0e6157
    67baca8
    8adee39
    2d5684e
  13. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Run 3 — private consumer validation through implementation readiness

    A fresh private repository created from the published template completed bootstrap, Cast, status, research, planning, and plan acceptance. Private organization and repository names are intentionally omitted.

    • Committed Cast state now survives activation; standalone v0.13.1 health passed with the committed roster and routing.
    • /squad plan bound every task to a certified active cast member and preserved the declared dependency ordering.
    • First /squad plan accept applied the expected lowercase squad:{member} labels and retained every dependency in issue bodies.
    • The acceptance run exposed two final UX defects: a flat plan created a duplicate epic/misparented one task, and its summary implied native dependency edges even though the installed gh-aw create_issue safe-output schema exposes hierarchy but not blockedBy.
    • Repeat /squad plan accept was idempotent and created no duplicate issues.
    • Commit 2d5684e2 now requires flat plans to create exactly one child per planned row under the originating issue, treats body references as the explicit fallback when native edges are unavailable, and forbids summaries from claiming hierarchy or edges that were not created.
    • Validation: 223 focused GH-AW tests passed; all four workflows compiled successfully in consumer layout under gh-aw strict mode (known slash-command/bot trigger warning only).

    Stopped before /squad implement, as planned.

  14. added a commit that references this issue on Aug 27, 2026
    9024878
  15. bradygaster commented on Aug 27, 2026

    @bradygaster
    CollaboratorAuthor

    Final post-merge consumer proof

    The merged lifecycle hardening was installed into the private consumer as all four immutable workflow references and passed strict compilation together.

    A fresh unphased three-item plan then validated the final acceptance behavior:

    • exactly three task issues were created, all directly parented to the originating intent;
    • each task received the expected certified squad:{member} label;
    • dependency rows resolved to the created issue references in task bodies;
    • the acceptance summary accurately reported the body-reference fallback because native blockedBy edges were unavailable; and
    • a repeated /squad plan accept created no duplicate issues.

    The run stopped before /squad implement. The authoritative journal closeout has also merged. No private organization or repository names are included here.

  16. added a commit that references this issue on Aug 28, 2026
    2028fbe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions