feat(ci): automated bug retro checklist on issue close #819

Problem Statement

When bugs are fixed and their issues closed, there is no systematic process to ask "why did this escape CI?" and "what should we change to prevent this class of bug?" Retros happen ad-hoc (if at all), and the lessons learned are lost. The casting/upgrade drift bug (#817) and SDK compatibility gap (#818) were both discovered by Brady during insiders testing — not by CI, not by code review, and not by automated tests.

Background

• Init/upgrade parity bug escaped because init and upgrade were tested in isolation — no cross-path validation existed
• SDK compatibility gaps escaped because all 150+ tests mock the SDK — zero runtime testing
• Both bugs were discoverable with existing tools but nobody asked the retro questions at the right time
• References: feat: Init/Upgrade Parity & SDK Compatibility Testing — PRD #817 (init/upgrade parity PRD), feat: @github/copilot-sdk Compatibility Testing & Breaking Change Detection — PRD #818 (SDK compatibility PRD)

Requirements

FR-1: Bug Retro Checklist on Issue Close

When a bug-labeled issue is closed, automatically post a retro checklist comment.

Acceptance criteria:

• GitHub Actions workflow triggers on issues.closed where label includes bug
• Posts a comment with retro checklist (see template below)
• Skippable — checklist says "Skip if trivial (cosmetic/typo)"
• Does NOT trigger on non-bug issues

FR-2: Retro Checklist Template

The comment should include:
``

🔄 Bug Retro Checklist

• Why did this escape CI? (missing test? untested code path? mock hid it?)
• Test added that would have caught it? (link PR)
• Is this a class of bug? (could the same pattern exist elsewhere?)
• CI/review change needed? (new lint rule? workflow gate? PR template checkbox?)
• Retro complete — check when all above are addressed or skipped

Auto-generated for closed bugs. Skip if trivial (cosmetic/typo). Uncheck "Retro complete" to reopen the retro.
``

FR-3: Severity-Based Retro Depth (Future)

Optional enhancement: use labels to determine retro depth.

Acceptance criteria:

• bug + severity:critical or security → full retro (all 4 questions mandatory)
• bug alone → lightweight retro (checklist, skippable)
• bug + cosmetic or typo → no retro posted

FR-4: Retro Tracking Dashboard (Future)

Track retro completion rates.

Acceptance criteria:

• Script or query that finds closed bug issues where "Retro complete" is unchecked
• Weekly summary of retro completion rate
• Stale retro reminder after 7 days unchecked

FR-5: Smart Pre-Fill via Copilot Skill (Future)

A Squad skill that reads the fix PR diff and pre-fills retro answers.

Acceptance criteria:

• Skill reads the linked PR's diff
• Identifies if a test was added (yes/no + file path)
• Identifies the bug category (mock-hidden, untested path, race condition, etc.)
• Pre-fills checklist with suggested answers
• Human reviews and confirms/edits

Non-Functional Requirements

NFR-1: Zero Noise for Trivial Bugs

The checklist must not create toil for typo/cosmetic fixes. Clear skip guidance.

NFR-2: No Mandatory Gating

Retro is advisory, not blocking. Closing an issue doesn't require completing the retro.

NFR-3: Works with @copilot PRs

When @copilot closes a bug issue via PR, the retro checklist still posts. The human reviewer should complete it.

Design Decisions

DD-1: Comment, Not Issue

Post as a comment on the closed issue, not as a new issue. Keeps context together.

DD-2: Checklist, Not Form

GitHub issue forms don't work on comments. Markdown checkboxes are universally supported and easy to complete.

DD-3: Phase 1 is Just the Workflow

Start with FR-1 + FR-2 (the workflow + template). FR-3/4/5 are future enhancements that build on the foundation.

Effort Estimates

Implementation

Phase 1 workflow:
yaml name: Bug Retro Prompt on: issues: types: [closed] jobs: retro: if: contains(github.event.issue.labels.*.name, 'bug') runs-on: ubuntu-latest steps: - uses: actions/github-script@v7 with: script: | await github.rest.issues.createComment({ issue_number: context.issue.number, owner: context.repo.owner, repo: context.repo.repo, body: '## 🔄 Bug Retro Checklist\n\n- [ ] **Why did this escape CI?**\n- [ ] **Test added that would have caught it?**\n- [ ] **Is this a class of bug?**\n- [ ] **CI/review change needed?**\n- [ ] **Retro complete**\n\n_Auto-generated. Skip if trivial._' });

References

• Init/upgrade parity retro (this session, 2026-04-04)
• feat: Init/Upgrade Parity & SDK Compatibility Testing — PRD #817 — Init/Upgrade Parity PRD
• feat: @github/copilot-sdk Compatibility Testing & Breaking Change Detection — PRD #818 — SDK Compatibility Testing PRD