Repository navigation
Description
Problem
Phase 1's carried-over-PR handling in the generated-skill template branches on
whether the open PR's checks are green:
If the PR is still valid ({{EXTERNAL_SIGNAL_LABEL}} green, no conflicts), first
confirm a Phase 4 self-review was actually posted …
and Phase 4 step 1 anchors the self-review rubric on gh pr checks <number> --watch.
Both treat the anchor as binary: green, or red-and-must-be-fixed. GitHub has a
third state that neither covers.
When a commit is pushed to a PR branch by an account whose workflow runs require
manual approval — a first-time contributor, or an agent account such as
copilot-swe-agent — the run is created with conclusion action_required and
executes nothing (0s duration). The workflow never ran; no job produced a result.
The failure mode is that this does not surface as a red check. It surfaces as an
absence:
gh pr checks <number>printsno checks reported on the '<branch>' branchgh pr checks <number> --watchreturns that same line immediately rather than
waiting, so a step written as "wait for the anchor to go green" returns at once
having established nothinggh run list --branch <branch>still shows asuccessrun — but on an older
head SHA, so a glance at the run history reads as a healthy branch
Nothing in Phase 1 or Phase 4 distinguishes "the anchor has not been run" from
"the anchor has no jobs configured for this branch". An unattended cycle can
therefore score the Phase 4 {{EXTERNAL_SIGNAL_LABEL}} rubric item off a run that
never executed, or read --watch returning instantly as confirmation.
Observed instance
Stephenson-Software/create-dev-loop PR #119. A merge-conflict resolution was
pushed to the branch by copilot-swe-agent as 26b149f; run 32547957642 on
that SHA completed action_required in 0s. gh pr checks 119 reported no checks
reported, while gh run list showed a success CI run two commits back on
9d7af16. The condition was resolved by pushing a further commit from an
already-permitted account, which retriggered the workflow with no approval gate
and produced a real green on the new head.
Suggested direction
Two changes, both small:
-
Phase 1 carried-over-PR handling — add a third branch alongside the
existing green / stale-or-conflicted cases and theTEMP:mid-revert check:
an anchor run in a not-executed state (action_required,waiting, or
stale) on the PR head is neither green nor red, and the PR is not ready for
the Phase 5 jump. Where the loop's own account can push, adding a commit or
re-running the workflow is enough to clear it; where it cannot, this is a
hand-off naming the approval gate. -
Phase 4 step 1 — state that the anchor must be confirmed against the
PR head SHA, and thatno checks reportedis not a pass. This parallels
the rule already added by PR Require unique scratch filenames and define the structurally-red-anchor path #121 for a build tool that exits 0 having
executed zero tests: a status that reports success without having run anything
verifies nothing. The same sentence covers both, and the existing
"Confirm the anchor actually executed tests" paragraph at Phase 3 is the
natural place to point at.
A useful check for either: compare gh run list --branch <branch> --json headSha,status,conclusion
against the head SHA from gh pr view <number> --json headRefOid, rather than
trusting the aggregate status line.
Research grounding
RESEARCH.md §1 (Self-critique without an external signal is unreliable,
confidence: high). This is the same failure the finding's PR #121 Implementations
entry already records — an anchor that carries no information being read as one
that does — reached by a third route. The existing entry covers a job that has
never succeeded on the base branch and a build tool that reports success having
executed zero tests; a run that was created but never permitted to execute is a
further instance, and would extend that entry rather than needing a new finding.
This issue body was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).
drafted by Claude on behalf of Daniel Stephenson
Activity
dmccoystephenson commentedon Nov 4, 2026
A second route to the same output was observed in a generated instance, and it is worth covering when this rule lands in the template.
On
Preponderous-Software/artificial-consciousness-simulation-framework, both workflows usepull_request: paths:filters, so a PR confined to files outside them gets no run at all.gh pr checks 222(and--watch, which returns at once) printedno checks reported on the 'docs/readme-visuals' branchand exited 1. Unlike the approval-gated case described above, no run object exists on any SHA, so thegh run listcomparison suggested here would find nothing.The two causes need different outcomes: a run that was never permitted is a missing signal, while a PR outside every filter never needed one. The distinguishing step used locally is to compare
git diff --name-only origin/main...HEADwith the workflow filters. All paths outside them is recorded as not applicable rather than green, and any path inside them is treated as a missing run. A short delay after a push can produce the same output, so a re-poll comes before that conclusion.The ACSF-specific form was added to that instance in Preponderous-Software/acsf-dev-loop#84. The generic rule was not folded in there and remains tracked here.
This comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).
drafted by Claude on behalf of Daniel Stephenson