Skip to content

fix(readiness): exclude workflow_run writer checks from CI evaluation - #2436

Merged
tamirdresher merged 2 commits into
devfrom
tamirdresher-microsoft-fix-readiness-exclude-workflow-run-check
Nov 9, 2026
Merged

tamirdresher merged 2 commits into
devfrom
tamirdresher-microsoft-fix-readiness-exclude-workflow-run-check

Conversation

@tamirdresher

Copy link
Copy Markdown
Collaborator

What's wrong

Since #2409, the impact, Post Repo Health findings, and heartbeat writers run as workflow_run jobs (squad-impact.yml, squad-repo-health-report.yml, squad-heartbeat.yml). GitHub attaches workflow_run check runs to the default-branch (dev) tip, not to the PR that triggered them.

squad-impact.yml and squad-repo-health-report.yml use cancel-in-progress: true, so two quick pushes on any PR leave a cancelled impact / Post Repo Health findings check run sitting on the dev tip. A binding or posting error leaves a failure there the same way.

scripts/pr-readiness.mjs -> checkCIStatus() treats cancelled and failure as failing CI, and SELF_CHECK_NAMES only excluded readiness and readiness-bind (the latter added in #2423 for exactly this reason). So every PR whose head is the dev tip, i.e. dev -> main promotion PRs, showed "check(s) failing: impact" based on other PRs' cancelled runs.

What changed

  • scripts/pr-readiness.mjs: new WORKFLOW_RUN_WRITER_CHECK_NAMES = ['impact', 'Post Repo Health findings', 'heartbeat'], spread into SELF_CHECK_NAMES, with a comment explaining why these land on the dev tip. Names were taken from the job name: (or job id when there is none) in each workflow.
  • test/pr-readiness.test.ts: for each excluded writer name, a cancelled, failure, and in_progress run no longer fails/pends CI; a real failure alongside an ignored writer run is still reported; constants test updated.
  • test/pull-request-workflow-split.test.ts: pins every job in squad-impact.yml, squad-repo-health-report.yml, and squad-heartbeat.yml to WORKFLOW_RUN_WRITER_CHECK_NAMES, mirroring the existing readiness-bind test, so a job rename breaks a test instead of readiness.
  • .copilot/skills/pr-lifecycle/SKILL.md: Check 9 now lists the excluded names and why.

Workflow concurrency settings are deliberately unchanged; the exclusion list is the chosen fix.

No changeset: the Changelog gate only covers packages/squad-(sdk|cli)/src|templates, .squad-templates/, and templates/.

How it was tested

  • npx vitest run test/pr-readiness.test.ts test/pull-request-workflow-split.test.ts test/workflow-run-pr-binding.test.ts test/repo-health-report.test.ts - passed (3 files, 217 tests).
  • Full npx vitest run test/ was started but not completed locally (gh-aw / packaging smoke suites run for 25+ minutes); relying on CI for that.

Relates to #2409, #2423.

#2409 moved the impact, Post Repo Health findings and heartbeat writers
onto workflow_run jobs. Their check runs attach to the default-branch
tip, not the PR head, and squad-impact.yml / squad-repo-health-report.yml
cancel superseded runs, so a quick push on any PR leaves a `cancelled`
check on dev. checkCIStatus() treats cancelled/failure as failing CI, so
every dev-to-main PR reported "check(s) failing: impact" based on
unrelated PRs' runs. readiness-bind was already excluded for this reason
(#2423); this adds the remaining workflow_run writer names.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 9, 2026 15:21
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

🟡 Impact Analysis — PR #2436

Risk tier: 🟡 MEDIUM

📊 Summary

Metric Count
Files changed 4
Files added 0
Files modified 4
Files deleted 0
Modules touched 3

🎯 Risk Factors

  • 4 files changed (≤5 → LOW)
  • 3 modules touched (2-4 → MEDIUM)

📦 Modules Affected

copilot-config (1 file)
  • .copilot/skills/pr-lifecycle/SKILL.md
scripts (1 file)
  • scripts/pr-readiness.mjs
tests (2 files)
  • test/pr-readiness.test.ts
  • test/pull-request-workflow-split.test.ts

This report is generated automatically for every PR. See #733 for details.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit fd4cb79

PR Scope: 🔧 Infrastructure

⚠️ 2 item(s) to address before review

Status Check Details
❌ Single commit 2 commits — consider squashing before review
✅ Not in draft Ready for review
✅ Branch up to date Up to date with dev
❌ Copilot review No Copilot review yet — it may still be processing
✅ Changeset present No source files changed — changeset not required
✅ Scope clean No .squad/ or docs/proposals/ files
✅ No merge conflicts No merge conflicts
✅ Copilot threads resolved No Copilot review threads
✅ CI passing All checks passing

Files Changed (4 files, +90 −6)

File +/−
.copilot/skills/pr-lifecycle/SKILL.md +2 −2
scripts/pr-readiness.mjs +25 −3
test/pr-readiness.test.ts +49 −0
test/pull-request-workflow-split.test.ts +14 −1

Total: +90 −6


This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the changes excluding unrelated writer checks from promotion PR readiness evaluation.

No open issues 🎉

What changed in this PR

Prevents unrelated workflow_run writer checks on the default-branch tip from incorrectly marking dev-to-main promotion PRs as failing or pending.

Changes:

  • Excludes impact, Post Repo Health findings, and heartbeat from readiness CI evaluation without changing workflow concurrency.
  • Adds regression tests for ignored writer runs, genuine CI failures, and workflow job-name consistency.
  • Documents the exclusions and why they are necessary.
File Description
test/​pull-request-workflow-split.test.ts Ensures writer job names remain in the CI exclusion lists.
test/​pr-readiness.test.ts Tests ignored writer statuses while preserving genuine failure reporting.
scripts/​pr-readiness.mjs Defines writer check names and adds them to readiness exclusions.
.copilot/​skills/​pr-lifecycle/​SKILL.md Explains the updated CI evaluation exclusions.

Review effort: Balanced (auto)

@tamirdresher
tamirdresher enabled auto-merge (squash) November 9, 2026 16:31
@tamirdresher
tamirdresher merged commit 245ff71 into dev Nov 9, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants