Skip to content

feat: PR architectural impact analysis — blast radius, module boundaries, reviewer routing #733

Closed

Description

@diberry

Problem Statement

When a PR lands on bradygaster/squad, there's no automated way to answer:

  • Which architectural boundaries does this PR cross?
  • Which modules (mapped by ownership in routing.md) are impacted?
  • What's the blast radius — if this breaks, what else breaks?
  • Does this PR introduce new dependencies between modules that didn't exist before?
  • Which squad members should review this, based on module ownership?

Today this is done by human judgment reading the diff. For AI agents making changes, there's no check at all — they don't know the boundaries they're crossing.

The Incident That Proves This Is Needed

Issue #631: @copilot accidentally deleted 361 source files in a single PR. An automated impact analysis would have:

  1. Immediately flagged the PR as CRITICAL risk (>50 files deleted, >8 modules touched)
  2. Posted an impact report as a PR comment with the blast radius
  3. Auto-suggested all module owners + Flight for review
  4. Blocked merge until Lead approval

Solution: Squad Impact — Automated PR Impact Analysis

Not a product feature. This is CI infrastructure for the Squad repo itself — a GitHub Action + reusable analysis script that runs on every PR to understand architectural impact.

High-Level Flow

PR opened on bradygaster/squad
    ↓
GitHub Action triggers (pull_request event)
    ↓
Action runs scripts/analyze-impact.mjs (Node.js script)
    ↓
Script analyzes diff against:
  - routing.md (module→agent ownership)
  - .github/CODEOWNERS (reviewer mapping)
  - package.json (workspace boundaries)
  - TEMPLATE_MANIFEST (critical files)
    ↓
Script outputs JSON impact report
    ↓
Action posts impact report as PR comment
    ↓
Action auto-requests reviewers + suggests squad:{agent} labels
    ↓
If CRITICAL: post merge warning

Outputs (PR Comment)

Example: PR #631 reanalyzed with this tool:

## 🔴 SQUAD IMPACT REPORT — PR #631

**Risk Tier:** CRITICAL (⚠️ Requires Lead approval before merge)

### 📦 Packages Affected (2 of 2)
- **squad-sdk** — 261 files deleted
- **squad-cli** — 100 files deleted

### 📁 Modules Touched (18 of 20)
- `src/adapter/` — 31 files (Primary: EECOM, Secondary: CAPCOM)
- `src/agents/` — 28 files (Primary: Procedures)
- `src/runtime/` — 24 files (Primary: GNC)
- `src/cli/` — 23 files (Primary: EECOM)
- `src/tools/` — 21 files (Primary: EECOM)
- ... (13 more modules)

### 💥 Impact Analysis

| Metric | Value | Status |
|--------|-------|--------|
| Files Changed | 361 | 🔴 CRITICAL (>50) |
| Modules Touched | 18 | 🔴 CRITICAL (>8) |
| Boundaries Crossed | 17 | 🔴 CRITICAL |
| New Dependency Edges | 12 | 🔴 HIGH |
| Critical Files Touched | 8 | 🔴 YES |

### 👥 Suggested Reviewers (by module ownership)
@squad/eecom (touched 5 modules) | @squad/gnc | @squad/procedures | @squad/flight (Architecture Lead - CRITICAL tier)

### 🏷️ Suggested Labels
- `squad:eecom`
- `squad:gnc`
- `squad:procedures`
- `impact:critical`
- `needs:flight-review`

Architecture

1. GitHub Action Workflow (.github/workflows/squad-impact.yml)

Triggers on PR, calculates impact, posts comment:

name: Squad Impact Analysis
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # needed for full diff
      
      - name: Analyze PR Impact
        run: node scripts/analyze-impact.mjs ${{ github.event.pull_request.number }}
        env:
          GH_TOKEN: ${{ github.token }}
          
      - name: Post Impact Report
        run: |
          gh pr comment ${{ github.event.pull_request.number }} \
            --body "$(cat impact-report.md)"
        env:
          GH_TOKEN: ${{ github.token }}
      
      - name: Request Reviewers
        run: node scripts/request-reviewers.mjs ${{ github.event.pull_request.number }}
        env:
          GH_TOKEN: ${{ github.token }}

2. Impact Analysis Script (scripts/analyze-impact.mjs)

A reusable Node.js script (NOT in an SDK module) that:

  1. Parses PR diff — gets list of added/modified/deleted files
  2. Maps files → modules — using routing.md module ownership table
  3. Detects boundaries — which modules are touched, which owners involved
  4. Calculates risk tier — based on criteria below
  5. Generates report — as Markdown, returns JSON for downstream use
  6. Auto-suggests labels — squad:{agent} labels based on module ownership
  7. Suggests reviewers — team members mapped from routing.md

Input:

// Usage:
// node scripts/analyze-impact.mjs <PR-number>

const impact = await analyzeImpactFromPR(prNumber, {
  routingFile: ".squad/routing.md",
  codeownersFile: ".github/CODEOWNERS",
  packageJsonFile: "package.json",
  workspacePath: "packages/*",
});

console.log(impact);
// Output: { riskTier, modulesTouched, reviewers, labels, report }

Output (JSON):

{
  "riskTier": "CRITICAL",
  "filesChanged": 361,
  "filesDeleted": 298,
  "modulesTouched": ["src/adapter/", "src/agents/", "src/runtime/", ...],
  "moduleCount": 18,
  "boundariesCrossed": 17,
  "newDependencyEdges": 12,
  "criticalFilesTouched": ["src/index.ts", "package.json", ...],
  "suggestedReviewers": ["@squad/eecom", "@squad/flight"],
  "suggestedLabels": ["squad:eecom", "impact:critical", "needs:flight-review"],
  "markdownReport": "## 🔴 SQUAD IMPACT REPORT..."
}

3. Data Sources (all already exist in the repo)

Source Location Provides
Module Ownership .squad/routing.md 20 directory → agent mappings
Critical File Manifest packages/squad-cli/src/cli/core/templates.ts (TEMPLATE_MANIFEST) Ownership flags for critical files
Package Boundaries package.json workspaces Package directories (packages/*)
TypeScript Projects tsconfig.json + packages/*/tsconfig.json Project boundaries
Code Owners (optional) .github/CODEOWNERS Human reviewer mapping

4. Repo-Level Integration (NOT an SDK export)

scripts/
├── analyze-impact.mjs           ← Main analysis engine
├── request-reviewers.mjs        ← POST /repos/.../pulls/.../requested_reviewers
├── impact-utils/
│   ├── parse-routing.mjs        ← Read & parse routing.md
│   ├── parse-manifest.mjs       ← Extract TEMPLATE_MANIFEST from templates.ts
│   ├── diff-parser.mjs          ← Parse `gh pr diff` output
│   └── risk-scorer.mjs          ← Calculate risk tier (logic below)
│
.github/workflows/
└── squad-impact.yml             ← Workflow definition (above)

Note: This is NOT a product feature export in packages/squad-sdk/. It's internal CI tooling that lives in scripts/ and .github/workflows/.


Risk Tier Calculation

Risk tier is calculated by combining multiple factors:

Tier Criteria
LOW • ≤1 module touched AND
• 0 new dependency edges AND
• ≤5 files changed AND
• No critical files touched
MEDIUM • 2–4 modules touched OR
• 1 new dependency edge OR
• 6–20 files changed
HIGH • 5–8 modules touched OR
• 2–3 new dependency edges OR
• 21–50 files changed OR
• Any critical file touched
CRITICAL • >8 modules touched OR
• >3 new dependency edges OR
• >50 files changed OR
• Mass deletion (>10 files deleted) OR
• Cross-package with new edges OR
• package.json / routing.md / .squad/* modified

Special Cases:

  • CRITICAL merge block: CRITICAL PRs require Lead (Flight) approval before merge
  • Auto-approve signal: LOW-tier PRs can be fast-tracked (optional)

Implementation Phases

Phase 1: Diff Parser + Risk Scorer (MVP)

  • scripts/analyze-impact.mjs — parse gh pr diff, map files → modules, calculate risk
  • scripts/impact-utils/ — helper modules (routing parser, risk scorer)
  • Test with manual runs: node scripts/analyze-impact.mjs 733
  • Owner: EECOM + Booster | Effort: 1 session

Phase 2: GitHub Action Integration

  • .github/workflows/squad-impact.yml — trigger on PR, call script, post comment
  • scripts/request-reviewers.mjs — auto-request reviewers based on module ownership
  • Test with new test PRs
  • Owner: Booster + EECOM | Effort: 1 session

Phase 3: Merge Gating (Optional)

  • Add environment protection rules — block CRITICAL PRs until Lead approves
  • Or: GitHub Action that adds "needs:flight-review" label (non-blocking but visible)
  • Owner: Flight | Effort: 0.5 session

Acceptance Criteria

Phase 1 (MVP) ✅ DONE when:

Phase 2 (Action Integration) ✅ DONE when:

  • .github/workflows/squad-impact.yml triggers on all PRs
  • Posts impact report as PR comment within 2 minutes
  • Auto-suggests squad:{agent} labels based on touched modules
  • Tests on 3 real PRs (varied impact): LOW, MEDIUM, CRITICAL
  • Reviewers can see at a glance which modules were touched and who owns them

Phase 3 (Merge Gating) ✅ DONE when (if implemented):

  • CRITICAL PRs are labeled needs:flight-review and/or blocked by environment rule
  • Flight can approve with branch protection override if intentional
  • False positives <5% (measure over 20 PRs)

Related Issues & References


Notes

  • Not intended as a product export. This is Squad repo-internal CI infrastructure.
  • Degrades gracefully: Minimum viable analysis = package.json + CODEOWNERS even without routing.md
  • Future enhancement: Could feed impact data into automated test selection (run only tests for touched modules)
  • Future enhancement: Could integrate with Slack/Teams to notify team leads when their modules change

Activity

  1. diberry commented on Apr 2, 2026

    @diberry
    ContributorAuthor

    ⚠️ Dependency: This issue now depends on #736 (repo registry). The registry provides the file→module→owner→criticality lookup that the impact analysis script consumes. Build order: #736 → #733.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or improvement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions