Skip to content

Phase 0c: Make gh aw compile non-skippable, add prompt-budget gate, replace string-assertion workflow tests #1732

Closed

Description

@bradygaster

Parent: #1729

Problem

Two testing gaps that make it unsafe to add a third workflow:

  1. gh aw compile silently skips. test/gh-aw-quality.test.ts (~line 614) skips the compile test when the gh aw binary is unavailable. In CI this means a workflow that does not compile can merge green.
  2. Implement-worker tests are string assertions. test/gh-aw-implement-workflow.test.ts is five regex/substring checks over markdown. It does not assert the worker's actual permission surface, tool allowlist, or trigger set — exactly the properties that matter for an authority-boundary workflow.

There is also no guard on the compiled prompt budget. Current source sum is roughly 74 KB against a ceiling near 100 KB; adding a review workflow narrows that margin.

Scope

test/gh-aw-quality.test.ts, test/gh-aw-implement-workflow.test.ts, and CI configuration.

Acceptance criteria

  • gh aw is installed in CI and the compile test fails rather than skips when the binary is missing.
  • A prompt-budget check warns at ≥ 85 KB and fails at the hard ceiling.
  • Implement-worker tests assert, structurally rather than by substring: declared triggers, presence of edit:, the file allowlist, protected-files: request_review, allowed-branches, issue-scoped concurrency, and max: 1 on PR creation.
  • The resulting test shape is reusable as the template for the review workflow's tests.

Notes

This is the issue that establishes the pattern the review workflow will mirror, so it is worth landing before Phase 1 rather than alongside it.

Scope Narrowed — Compile Gate Only (2026-08-20 triage)

FIDO's triage split this issue into three verdicts:

  • SHIP-NOW (this issue): Add gh extension install github/gh-aw to squad-ci.yml's test job so the compile test actually runs in CI.
  • CLOSED — prompt-budget gate: Already done. test/gh-aw-quality.test.ts:637-669 enforces the 100 KB ceiling and >5 KB headroom guard. Ships with every npm test.
  • CLOSED — string-assertion replacement: Too vague to action. Tests are already substantially structural (parser helpers throughout: extractFrontmatter, extractSafeOutputs, extractModeTable, extractInlineSkills).

This issue now tracks the compile gate only.

Goal and Success Criteria

Goal: Make gh aw compile an actual CI gate — one that fails the squad-ci.yml test job when any squad workflow fails to compile, instead of silently skipping because gh aw is never installed in that job.

SC-1 — Install command present in CI job.
squad-ci.yml's test job contains gh extension install github/gh-aw before the step that invokes the compile test. Fails if that line is absent from the job.

SC-2 — Compile test no longer skips in CI.
test/gh-aw-quality.test.ts:978's it.skipIf(!ghAwAvailable) guard resolves to a running test (not a skip) in a CI run after this fix. Fails if the test still appears as skipped in CI logs.

SC-3 — A broken workflow fails the gate.
A workflows/squad.md with a deliberately introduced compile error (missing required field, invalid YAML, broken skill block) causes gh aw compile to exit non-zero and the CI test job to report a failure. Fails if a workflow with a compile error merges green through CI.

Activity

  1. added
    squadSquad triage inbox — Lead will assign to a member
    on Aug 17, 2026
  2. github-actions commented on Aug 17, 2026

    @github-actions
    Contributor

    📋 Assigned to FIDO (Quality Owner)

    Issue: #1732 — Phase 0c: Make gh aw compile non-skippable, add prompt-budget gate, replace string-assertion workflow tests

    FIDO will pick this up in the next Copilot session.

    For Copilot coding agent: If enabled, this issue will be worked automatically.
    Otherwise, start a Copilot session and say:
    FIDO, work on issue #1732

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    gh-awGitHub Agentic Workflows integrationpriority:p1This sprintsquadSquad triage inbox — Lead will assign to a membertriagedIssue has been triagedtype:choreMaintenance, refactoring, cleanupwave:1-nextWave 1 — immediate next sprint

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions