Skip to content

PRD: CI Template Overhaul — Performance & Safety for Shipped Workflows #886

Closed

Description

@diberry

PRD: CI Template Overhaul — Performance & Safety for Shipped Workflows

v2 — Updated with team review feedback from ⚙️ Booster, 🔒 RETRO, 🏗️ Flight, 🧪 FIDO

Problem Statement

Squad ships 11 GitHub Actions workflow templates (4 auto-copied via squad init, 7 opt-in) that run on other people's repositories consuming their Actions minutes and operating with their permissions. An audit reveals systemic performance waste and safety gaps across these templates.

Impact: Every Squad user pays the cost of these inefficiencies on every workflow run. With ~50+ active Squad installations, even small per-run savings compound significantly.

Migration note: Existing Squad users have static copies of these workflows. squad upgrade re-syncs workflow files from templates. ⚠️ upgrade unconditionally overwrites — users with customized workflows lose changes. Document this prominently in release notes and add a pre-upgrade diff warning in a future release.

Current State

Template Inventory

Template Auto-Copied Purpose Triggers Has setup-node?
squad-heartbeat.yml ✅ Issue triage, @copilot routing issues, pull_request, workflow_dispatch ❌
squad-triage.yml ✅ Keyword-based issue triage issues (labeled) ❌
squad-issue-assign.yml ✅ Member/copilot assignment issues (labeled) ❌
sync-squad-labels.yml ✅ Label sync from team.md push (.squad/team.md), workflow_dispatch ❌
squad-ci.yml ❌ opt-in Build + test pull_request, push ✅
squad-docs.yml ❌ opt-in Docs build + deploy workflow_dispatch, push ✅ (cached)
squad-preview.yml ❌ opt-in Preview validation push ✅
squad-promote.yml ❌ opt-in Branch promotion workflow_dispatch ❌
squad-release.yml ❌ opt-in GitHub Release creation push (main) ✅
squad-insider-release.yml ❌ opt-in Insider release tagging push (insider) ✅
squad-label-enforce.yml ❌ opt-in Label mutual exclusivity issues (labeled) ❌

Audit Summary (v2 — corrected per team review)

Category Finding Affected Templates
🔴 No npm caching 4 templates with setup-node missing cache: npm squad-ci, squad-preview, squad-release, squad-insider-release
🔴 No timeouts 0/11 templates have timeout-minutes ALL
🔴 Elevated token in user script ralph-triage.js runs user-customizable JS with GITHUB_TOKEN (issues:write) in env squad-heartbeat
🟠 No concurrency 10/11 templates missing concurrency groups All except squad-docs.yml
🟠 Actions not SHA-pinned All use floating @v4/@v7 tags (26 references) ALL
🟠 COPILOT_ASSIGN_TOKEN undocumented Users don't know what scopes to set heartbeat, issue-assign
🟠 GITHUB_TOKEN fallback unsafe heartbeat falls back to wider token when PAT absent squad-heartbeat
🟡 Label abuse vector squad:copilot trigger needs actor permission check triage, issue-assign, label-enforce
🟡 No path filters on CI CI triggers on all file changes squad-ci

Overly broad PR event types — REMOVED: Both squad-ci.yml and squad-heartbeat.yml already have explicit types: filters. (Corrected by Booster)


Proposed Changes

Phase 1 — Performance + Critical Safety (Low-Medium Risk, High Impact)

Goal: Reduce Actions minutes by ~40-60% and close supply chain gaps on write-access workflows.

1.1 Add npm caching to templates with setup-node

Files: squad-ci.yml, squad-preview.yml, squad-release.yml, squad-insider-release.yml (4 templates)

⚠️ npm-only: cache: npm requires package-lock.json. Users on pnpm/yarn must adjust to cache: pnpm or cache: yarn. Document this requirement with an inline comment: # REQUIRES: package-lock.json (npm). For pnpm/yarn, change cache value.

- uses: actions/setup-node@v4
  with:
    node-version: 22
    cache: npm   # REQUIRES: package-lock.json. For pnpm: cache: pnpm. For yarn: cache: yarn.

Impact: Saves 2-5 min per run.

1.2 Add timeout-minutes to every job

Files: All 11 templates, per-workflow mapping:

Tier Minutes Workflows
Light 10 squad-triage, squad-issue-assign, squad-label-enforce, sync-squad-labels
Medium 15 squad-ci, squad-preview, squad-promote, squad-heartbeat, squad-release, squad-insider-release
Heavy 20 squad-docs (VitePress cold builds can exceed 10 min)

Impact: Prevents runaway jobs. Safety net, not optimization.

1.3 Add concurrency groups (with explicit cancel-in-progress per workflow)

Workflow Group cancel-in-progress Rationale
squad-ci ${{ github.workflow }}-${{ github.ref }} true Newer commit supersedes
squad-preview ${{ github.workflow }}-${{ github.ref }} true Same
squad-release ${{ github.workflow }}-${{ github.ref }} false Never cancel a running release
squad-insider-release ${{ github.workflow }}-${{ github.ref }} false Same
squad-promote ${{ github.workflow }} (no ref) false Cancel = corrupted branches
squad-triage ${{ github.workflow }}-${{ github.event.issue.number }} false Let triage finish cleanly
squad-issue-assign ${{ github.workflow }}-${{ github.event.issue.number }} false Same
squad-label-enforce ${{ github.workflow }}-${{ github.event.issue.number }} false Same
squad-heartbeat ${{ github.workflow }} false Don't kill triage midway
sync-squad-labels ${{ github.workflow }} true Latest team.md wins

1.4 Add path filters to squad-ci.yml (using dorny/paths-filter)

⚠️ Do NOT use paths-ignore — it causes required status checks to hang in "pending" state (discovered in #872). Use the dorny/paths-filter action + conditional jobs instead.

jobs:
  changes:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    outputs:
      src: ${{ steps.filter.outputs.src }}
    steps:
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            src:
              - '!docs/**'
              - '!*.md'
              - '!.squad/*.md'
              - '!.squad/decisions/**'
  
  build:
    needs: changes
    if: needs.changes.outputs.src == 'true'
    ...

Note: Only .squad/*.md and .squad/decisions/ are filtered — .squad/skills/, .squad/agents/, and other subdirs still trigger CI (they may contain testable content).

1.5 SHA-pin actions in contents: write workflows

Files: squad-release.yml, squad-insider-release.yml, squad-promote.yml

These workflows can push to main, create tags, and modify branches. A supply chain compromise of actions/checkout@v4 here is catastrophic. SHA-pin these first, before the remaining read-only workflows in Phase 2.


Phase 2 — Safety Hardening (Medium Risk, High Impact)

Goal: Close remaining supply chain gaps, fix token issues, harden label triggers.

2.1 SHA-pin remaining action references

Pin all remaining action references in read-only workflows (CI, docs, triage, label, heartbeat, labels-sync). Include version comment:

- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2

2.2 Document COPILOT_ASSIGN_TOKEN (corrected scope)

⚠️ Use fine-grained PAT, NOT classic PAT with repo scope. Classic repo grants full read/write to all private repos — catastrophic blast radius if compromised.

# COPILOT_ASSIGN_TOKEN: A fine-grained GitHub PAT with 'Issues: Write' permission.
# Used to assign the @copilot coding agent to issues. Optional — if not set,
# @copilot assignment is skipped.
# Create at: Settings > Developer settings > Fine-grained tokens
# Required permission: Issues (Read and Write) on this repository only.

2.3 Remove GITHUB_TOKEN fallback in heartbeat

Current (unsafe):

github-token: ${{ secrets.COPILOT_ASSIGN_TOKEN || secrets.GITHUB_TOKEN }}

Fixed:

- name: Ralph — Assign @copilot issues
  if: success() && secrets.COPILOT_ASSIGN_TOKEN != ''
  uses: actions/github-script@v7
  with:
    github-token: ${{ secrets.COPILOT_ASSIGN_TOKEN }}

2.4 Remove GITHUB_TOKEN from ralph-triage.js env

squad-heartbeat.yml runs ralph-triage.js (user-customizable script) with GITHUB_TOKEN in the environment. This is the highest-risk pattern in the template set — arbitrary code execution with elevated permissions.

Fix: Remove GITHUB_TOKEN from the run: step's env: block. If the script needs API access, move that logic into the controlled github-script step.

2.5 Add label abuse guards

Add bot-loop filter + actor permission check for copilot assignment:

jobs:
  triage:
    if: |
      github.event.label.name == 'squad' &&
      github.actor != 'github-actions[bot]' &&
      github.actor != 'dependabot[bot]'

For squad:copilot assignment specifically, add an actor permission check before the copilot-assign step.

2.6 Scope id-token: write in squad-docs.yml

Move id-token: write from workflow-level to deploy job only:

jobs:
  build:
    permissions:
      contents: read
  deploy:
    permissions:
      pages: write
      id-token: write

Phase 3 — Quality of Life (Low Risk, Low Effort)

3.1 Add workflow_dispatch to squad-ci.yml

3.2 persist-credentials: false for read-only checkouts

Files: squad-triage, squad-label-enforce, squad-issue-assign (read-only checkouts only)

⚠️ Do NOT apply to: squad-release, squad-insider-release, squad-promote — these do git push and need persisted credentials.

3.3 Ship Dependabot template

Ship .github/dependabot.yml for automated SHA update PRs. Document that this must be merged with any existing Dependabot config, not overwritten.


Estimated Impact

Metric Current After Phase 1 After Phase 2
Minutes per CI run 15-20 min 8-12 min 8-12 min
Minutes per triage event 2-4 min 2-3 min 2-3 min
Supply chain (write-access) 🔴 Floating tags 🟢 SHA-pinned 🟢 SHA-pinned
Supply chain (read-only) 🔴 Floating tags 🔴 Floating tags 🟢 SHA-pinned
Runaway job protection None 10-20 min cap 10-20 min cap
Duplicate run waste ~30% of runs ~5% ~5%
Token documentation Undocumented Undocumented Fully documented
ralph-triage.js risk 🔴 Elevated token 🔴 Elevated token 🟢 Sandboxed

Per-user savings (Phase 1): ~200-500 minutes/month (conservative floor for active repos).


Implementation Process

  1. All edits go to .squad-templates/workflows/ ONLY (canonical source)
  2. Run node scripts/sync-templates.mjs --sync to propagate to 3 mirror directories
  3. Run npm test to verify template-sync.test.ts passes (byte-identical × 4 dirs)
  4. Add structural tests per testing plan below
  5. Each phase ships as a separate PR with changeset

Testing Plan (from FIDO review)

Add to workflows.test.cjs or new template-ci-hardening.test.cjs:

  • All templates have timeout-minutes on every job
  • All templates have a concurrency: block
  • Templates with setup-node include cache: npm
  • Release/promote workflows have cancel-in-progress: false
  • squad-ci.yml uses dorny/paths-filter (not paths-ignore)
  • Label-triggered workflows scope concurrency to github.event.issue.number
  • Label-triggered templates do NOT have spurious cache: npm
  • YAML structural validation — all 11 templates parse without errors

Non-Goals


Open Questions (resolved)

  1. ✅ Dependabot: Ship both template AND docs (complementary)
  2. ✅ COPILOT_ASSIGN_TOKEN scopes: Fine-grained PAT with Issues: Write only (NOT classic repo)
  3. ✅ Timeout values: 10/15/20 min tiers mapped to specific workflows

Success Criteria

  • All 11 templates have timeout-minutes on every job (10/15/20 tiers)
  • 4 templates with setup-node have cache: npm with npm-only documentation
  • All 11 templates have concurrency groups with correct cancel-in-progress
  • squad-ci.yml uses dorny/paths-filter (not paths-ignore)
  • contents: write workflows have SHA-pinned actions (Phase 1)
  • All remaining actions SHA-pinned (Phase 2)
  • COPILOT_ASSIGN_TOKEN documented as fine-grained PAT with Issues: Write
  • GITHUB_TOKEN fallback removed from heartbeat
  • ralph-triage.js no longer has GITHUB_TOKEN in env
  • Label-triggered workflows have bot-loop + permission guards
  • id-token: write scoped to deploy job in squad-docs.yml
  • persist-credentials: false on read-only checkouts (NOT release/promote)
  • Structural tests validate all the above
  • Template sync tests pass (byte-identical across 4 directories)

Activity

  1. added
    squadSquad triage inbox — Lead will assign to a member
    on Apr 6, 2026
  2. github-actions commented on Apr 6, 2026

    @github-actions
    Contributor

    🏗️ Squad Triage — Flight (Lead)

    Issue: #886 — PRD: CI Template Overhaul — Performance & Safety for Shipped Workflows
    Assigned to: FIDO (Quality Owner)
    Reason: Issue relates to testing/quality work

    @copilot evaluation: 🔴 Not suitable — issue involves work outside the coding agent's capability profile.

    Team roster:

    • Flight (Lead) → label: squad:flight
    • Procedures (Prompt Engineer) → label: squad:procedures
    • EECOM (Core Dev) → label: squad:eecom
    • FIDO (Quality Owner) → label: squad:fido
    • PAO (DevRel) → label: squad:pao
    • CAPCOM (SDK Expert) → label: squad:capcom
    • CONTROL (TypeScript Engineer) → label: squad:control
    • Surgeon (Release Manager) → label: squad:surgeon
    • Booster (CI/CD Engineer) → label: squad:booster
    • GNC (Node.js Runtime) → label: squad:gnc
    • Network (Distribution) → label: squad:network
    • RETRO (Security) → label: squad:retro
    • INCO (CLI UX & Visual Design) → label: squad:inco
    • GUIDO (VS Code Extension) → label: squad:guido
    • Telemetry (Aspire & Observability) → label: squad:telemetry
    • VOX (REPL & Interactive Shell) → label: squad:vox
    • DSKY (TUI Engineer) → label: squad:dsky
    • Sims (E2E Test Engineer) → label: squad:sims
    • Handbook (SDK Usability) → label: squad:handbook
    • Ralph (Work Monitor) → label: squad:ralph
    • @copilot (Coding Agent) → label: squad:copilot

    To reassign, remove the current squad:* label and add the correct one.

  3. diberry commented on Apr 6, 2026

    @diberry
    ContributorAuthor

    Team Review — v2 Updates

    Reviewers: ⚙️ Booster (CI/CD), 🔒 RETRO (Security), 🏗️ Flight (Lead), 🧪 FIDO (Quality)
    Verdict: All 4 REQUEST CHANGES — blockers identified and incorporated below.

    🔴 Blockers Fixed in v2

    # Issue Resolution
    B1 cache: npm breaks pnpm/yarn users Corrected: only 4 templates need caching, documented npm-only requirement
    B2 paths-ignore known-broken (hangs required checks) Replaced with dorny/paths-filter + conditional jobs pattern
    B3 "Overly broad PR event types" is FALSE Removed from audit — both workflows already have types: filters
    B4 "10/11 missing cache" misleading Corrected to "4 templates with setup-node need caching"
    B5 No migration story Added: squad upgrade re-syncs workflows, document customization clobber
    B6 cancel-in-progress not specified per type Added per-workflow concurrency table with explicit true/false
    B7 No testing plan Added: structural assertions for timeouts, concurrency, caching
    B8 Template sync not in impl plan Added: all edits to .squad-templates/ then node scripts/sync-templates.mjs --sync
    B9 COPILOT_ASSIGN_TOKEN repo scope too broad Fixed: specify fine-grained PAT with issues: write only
    B10 GITHUB_TOKEN fallback in heartbeat unsafe Fixed: remove fallback, skip step if token absent
    B11 ralph-triage.js runs with elevated token Added as new Phase 2 item: remove GITHUB_TOKEN from run env

    🟠 Suggestions Incorporated

    • SHA pinning for contents: write workflows moved to Phase 1 (Booster + RETRO)
    • Timeout values revised to 10/15/20 min tiers with per-workflow mapping (Flight + Booster + FIDO)
    • .squad/** path filter narrowed to .squad/*.md + .squad/decisions/ only (FIDO)
    • persist-credentials: false explicitly excludes release/promote workflows (Booster)
    • id-token: write in squad-docs.yml scoped to deploy job only (RETRO)
    • squad:copilot trigger gets actor permission check (RETRO)
    • Comprehensive testing plan from FIDO added to success criteria

    Corrected Audit Summary

    Category Finding Affected Templates
    🔴 No npm caching 4 templates with setup-node missing cache: npm squad-ci, squad-preview, squad-release, squad-insider-release
    🔴 No timeouts 0/11 templates have timeout-minutes ALL
    🔴 ralph-triage.js elevated token Runs user JS with GITHUB_TOKEN (issues:write) in env squad-heartbeat
    🟠 No concurrency 10/11 templates missing concurrency groups All except squad-docs.yml
    🟠 Actions not SHA-pinned All use floating @v4/@v7 tags ALL (26 references)
    🟠 COPILOT_ASSIGN_TOKEN undocumented Users don't know scopes; PRD originally over-scoped to repo heartbeat, issue-assign
    🟠 GITHUB_TOKEN fallback unsafe heartbeat falls back to wider token when PAT absent squad-heartbeat
    🟡 Label abuse vector squad:copilot trigger needs permission check triage, issue-assign, label-enforce
    🟡 No path filters on CI CI triggers on all file changes squad-ci
    🟡 Overly broad PR event types REMOVED — both workflows already have types filters N/A

    Revised Phase Plan

    Phase 1 — Performance + Critical Safety (merged SHA pinning for write-access workflows)

    1. Add cache: npm to 4 templates (with npm-only requirement documented)
    2. Add timeout-minutes per workflow (10/15/20 min tiers)
    3. Add concurrency groups with per-workflow cancel-in-progress table
    4. Replace paths-ignore with dorny/paths-filter + conditional jobs
    5. SHA-pin actions in contents: write workflows (release, insider-release, promote)

    Phase 2 — Safety Hardening

    1. SHA-pin remaining action references (read-only workflows)
    2. Document COPILOT_ASSIGN_TOKEN as fine-grained PAT issues: write
    3. Remove GITHUB_TOKEN fallback in heartbeat — skip step if token absent
    4. Remove GITHUB_TOKEN from ralph-triage.js run: env
    5. Add label abuse guards (bot-loop filter + actor permission check for copilot assignment)
    6. Scope id-token: write to deploy job in squad-docs.yml

    Phase 3 — QoL

    1. Add workflow_dispatch to squad-ci.yml
    2. persist-credentials: false for read-only checkouts (explicitly NOT release/promote)
    3. Ship Dependabot template (merge-required, not overwrite)

    Revised Per-Workflow Concurrency Table

    Workflow Group cancel-in-progress Why
    squad-ci workflow-ref true Newer commit supersedes
    squad-preview workflow-ref true Same
    squad-docs pages (already correct) true Already correct
    squad-release workflow-ref false Never cancel a release
    squad-insider-release workflow-ref false Same
    squad-promote workflow (no ref) false Cancel = corrupted branches
    squad-triage workflow-issue false Let triage finish cleanly
    squad-issue-assign workflow-issue false Same
    squad-label-enforce workflow-issue false Same
    squad-heartbeat workflow false Don't kill triage midway
    sync-squad-labels workflow true Latest team.md wins

    Revised Timeout Table

    Tier Minutes Workflows
    Light 10 squad-triage, squad-issue-assign, squad-label-enforce, sync-squad-labels
    Medium 15 squad-ci, squad-preview, squad-promote, squad-heartbeat, squad-release, squad-insider-release
    Heavy 20 squad-docs (VitePress cold builds)

    Implementation Process

    1. All edits go to .squad-templates/workflows/ ONLY
    2. Run node scripts/sync-templates.mjs --sync to propagate to 3 mirror directories
    3. Run npm test to verify template-sync.test.ts passes
    4. Add structural tests per FIDO's testing plan
    5. Each phase ships as a separate PR with changeset
  4. added 3 commits that reference this issue on Apr 6, 2026
    c114b3e
    1b7feef
    697274a
  5. diberry commented on Apr 10, 2026

    @diberry
    ContributorAuthor

    Closing — descoping high-touch feature work to focus on existing PRs and low-touch fixes.

  6. added a commit that references this issue on Apr 13, 2026
    79eab41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    go:needs-researchNeeds investigationsquadSquad triage inbox — Lead will assign to a member

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions