Skip to content

annotation-gate: name the runs a deferred gate waits on - #2152

Open
d-morrison wants to merge 6 commits into
mainfrom
claude/project-thread-951mnp
Open

d-morrison wants to merge 6 commits into
mainfrom
claude/project-thread-951mnp

Conversation

@d-morrison

@d-morrison d-morrison commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #2151.

With defer-while-running, the annotation gate posts nothing while any other run on the head is still queued or in progress.
When the self-hosted pool is full, that wait can last hours.
Until now the log said only how many runs were unfinished, so a gate that was held back looked like one that had never posted.
That is what happened on Morrison-Lab/lds on 2026-11-20: the gates were held, not lost.

Changes

  • annotation-gate/annotation_gate.py
    • New waiting_lines() builds one line per unfinished run, giving its name, status, minutes since creation, and link.
      A new helper, one_line(), collapses whitespace in the name and link, so a line break in a run name cannot split a job-summary list item or start new Markdown.
    • New command_data() escapes %, carriage returns and newlines, in that order, so a run name cannot break or inject a workflow command.
    • New report_deferral() replaces the bare count.
      The log line still gives only the count.
      The runs are named in a ::notice title=Annotation gate deferred:: and in a list appended to $GITHUB_STEP_SUMMARY when that file is set.
    • The gate's decision is unchanged: it defers on the same condition and exits the same way.
  • annotation-gate/tests/test_annotation_gate.py has five new tests:
    • a deferred gate names its waiting run in the job summary and in exactly one notice line;
    • the notice escapes % in a run name, and a newline in the name does not start a new command;
    • command_data() escapes %, CR and LF;
    • a run name or URL holding line breaks stays on one line;
    • the minutes-since-creation arithmetic is pinned.
  • changelog.d/annotation-gate-deferral-summary.changed.md: a changelog fragment.

All 78 annotation-gate tests pass locally (python3 -m pytest annotation-gate/tests -q prints 78 passed).

🤖 Generated with Claude Code

https://claude.ai/code/session_019xfc9nLowW5puRpJrWhahm

A gate deferred by defer-while-running printed only a count, so a gate
held behind a full self-hosted pool looked like one that never posted.
It now emits a notice and a step-summary list with each waiting run's
name, status, age in minutes and link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9nLowW5puRpJrWhahm
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator Author

Annotation dispositions for head d6004d3:

None of the annotations on this head needs a fix in this PR.

I read all 109 runs with scripts/batch-read-annotations.py and diffed their messages against main's tip, e77793f, which has the same 109 runs. Every message also appears on main except three, and all three are false positives:

  • Merge <sha> (Merge keeping one side of the fixture) dropped 1 line(s) (failure): this is the merge-drops selftest's negative control. The fixture merge sha is regenerated on every run, so the text never matches main's exactly. The job passed.
  • waited 1s for the R install lock /tmp/tmp.<random>/lock (failure): this is the R install-lock self-test. The temp path is random on every run.
  • Antigravity Agent encountered API rate limit or transient error ... Retrying (warning): a transient retry in the agy review that then succeeded. It is unrelated to this diff.

Everything else matches main, including the selftest negative controls (dead.md 404, divergent #def-variance, duplicate roxygen, the code.R lint), the Node 20 notices, and the review job's workflow-validation skip.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

A run name holding %, CR or LF would end the ::notice command early or start
a new one. Escape the data part as GitHub's toolkit does. Found by a
cross-harness review (iceman-contributor-free) of gha#2152.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9nLowW5puRpJrWhahm

Copy link
Copy Markdown
Collaborator Author

Cross-harness review of head d6004d3: OpenCode iceman-contributor-free, reading the full diff with no tools. Verdict: NOT_CLEAN, with one finding.

  • annotation-gate/annotation_gate.py:192: the ::notice data was not escaped, so a run name or URL holding %, CR or LF could end the command early or inject another one.

Fixed in baa2847. The notice data is now escaped as GitHub's toolkit does it (% to %25, CR to %0D, LF to %0A), and a new test pins this. All 76 annotation-gate tests pass locally.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator Author

Annotation dispositions for head baa2847:

All checks on this head passed. I read the warning and failure annotations with summarize-pr-annotations.sh against main's tip e77793f. Every one that differs from main is a selftest negative control or a transient retry. None comes from this diff, which changes only annotation-gate/.

  • False positive, selftest negative controls (failure level):

    • the duplicate with: key in docs/bad.qmd;
    • the merge-drops fixture;
    • ci-quarto-links-fixture/orphan-*.png;
    • the R install-lock wait.

    Each job asserts that its fixture fails and then passes. The paths differ from main only because the temporary directories and run ids differ.

  • False positive, selftest warnings:

    • the three Could not save the PR diff cases from the save-pr-diff tests;
    • the orphan-image warning;
    • the triage-backlog ceiling of 0, a fixture threshold.
  • Transient: one Antigravity rate-limit retry, attempt 2 of 3, which then succeeded.

  • Routine notices: dry-run comment bodies, v3 is up to date, and the no-#def--titles notice.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

Cross-harness review of head baa2847 by opencode (iceman-contributor-free, no tools, diff only): NOT_CLEAN, with one finding.

  • Finding: annotation-gate/annotation_gate.py:178 parses created_at with a fixed %Y-%m-%dT%H:%M:%SZ format, so fractional seconds would raise ValueError.
  • Rebuttal, no change: created_at comes from the Actions workflow-runs REST response, and GitHub returns it as ISO 8601 with whole seconds and a Z suffix, for example 2026-11-20T09:06:03Z. The suite's fixtures use that same shape.
  • Why no guard: catching ValueError here would hide a real change in the API's format. An error is the fail-fast outcome the repo prefers.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@d-morrison

Copy link
Copy Markdown
Collaborator Author

Cross-model review (agy, gemini-4-hydrogen-high) of head baa2847:

Summary

This PR adds waiting_lines(), command_data(), and report_deferral() in annotation-gate/annotation_gate.py, updates gate_commit() to call report_deferral() instead of only printing the deferred run count, adds unit tests in annotation-gate/tests/test_annotation_gate.py, and adds a changelog entry in changelog.d/annotation-gate-deferral-summary.changed.md. The core deferral logic, age arithmetic, and workflow-command escaping (%, \r, \n in order) are sound, but there are mismatches between the PR description/docstring and the diff.

Findings

  1. PR description mismatches the test diff (annotation-gate/tests/test_annotation_gate.py)

    • The PR description states that two tests are added ("One test asserts that the summary and the notice name every waiting run. / One test pins the minutes-since-creation arithmetic.") and does not mention command_data() or escaping.
    • The diff actually adds three tests, omitting test_a_deferral_notice_escapes_run_names from the summary.
    • test_a_deferred_gate_names_the_runs_it_waits_on_in_the_summary tests only a single waiting run (queued) and only asserts the ::notice title=Annotation gate deferred:: prefix on stdout—not that the notice itself names every waiting run (the notice content is only asserted in test_a_deferral_notice_escapes_run_names).
  2. Docstring and PR description overstate what is printed to the log (annotation-gate/annotation_gate.py)

    • report_deferral()'s docstring says Name the runs a deferred gate waits on, in the log, a notice, and the job summary. and the PR description says New report_deferral() replaces the bare count.
    • In report_deferral(), the log line (print(f"annotation-gate: deferred, {len(waiting)} run(s) on {head_sha[:7]} still queued or in progress; the last to finish gates the commit.")) is unchanged and still reports only the bare count; the waiting runs are named only in the ::notice workflow command and $GITHUB_STEP_SUMMARY.

Verdict

The implementation and tests are functionally solid, but the PR description and report_deferral() docstring do not accurately match the diff.

VERDICT: NOT CLEAN

Reviewed-Commit: baa2847

Posted by Claude Code (AI agent) --- not written by a human.

The log line still gives only a count; the notice and the job summary name
the runs. The first deferral test now asserts the notice names the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9nLowW5puRpJrWhahm
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator Author

Annotation dispositions for head 75c315a:

  • Failures: 317 on this head and 317 on main's tip e77793f (this PR's merge base), from the same checks. They are _selftest.yml negative controls (the _ci-* fixtures, Process completed with exit code 1. on the continue-on-error steps, the version-check contract fixture) and whole-tree warn-only scans of website/reference/*.qmd. False positive for this PR: none comes from a line it changes.
  • The failure lines that differ from main differ only by run-specific values: the merge-drops fixture's merge SHA ("Merge keeping one side of the fixture"). Same control, false positive.
  • Warnings: 130 here, 139 on main. yamllint comment spacing, actionlint and zizmor summaries, the credential-shape fixture, the localhost model endpoint, save-pr-diff failure cases, the open-issue ceiling (count differs by run: 220 now), and the antigravity retry are all pre-existing or selftest controls. False positive.
  • No annotation points at a file this PR changes (annotation-gate/annotation_gate.py, its test file, changelog.d/annotation-gate-deferral-summary.changed.md). The five AI-tell warnings on other changelog.d/ fragments are whole-tree and on main too.
  • Notices: routine, grouped.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

A run name or URL holding a line break was escaped for the ::notice
command but written raw into GITHUB_STEP_SUMMARY, so it could split a
list item or start new Markdown there. waiting_lines now collapses
whitespace in the name and link, which covers the summary and the
notice alike; command_data still escapes % and gets its own test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9nLowW5puRpJrWhahm

Copy link
Copy Markdown
Collaborator Author

Review replies, through head 6cbb91d:

  • agy NOT CLEAN on baa2847, annotation-gate/annotation_gate.py report_deferral docstring: it said the log line names the runs, but the log line only counts them. Addressed in 75c315a. The docstring now says the log counts the runs and the notice and summary name them.
  • agy, annotation-gate/tests/test_annotation_gate.py, first deferral test: it asserted only the notice prefix. Addressed in 75c315a. It now asserts exactly one notice line, carrying the run's name, status and age.
  • Cross-harness review of 75c315a, annotation-gate/annotation_gate.py:208: a line break in a run name or URL was escaped for ::notice but written raw to the job summary. Addressed in 6cbb91d. waiting_lines now collapses whitespace in the name and link, so both outputs get one line per run. test_waiting_lines_keep_a_run_name_on_one_line fails on 75c315a and passes on 6cbb91d. command_data keeps its own test for % and CR/LF. python3 -m pytest annotation-gate/tests -q reports 78 passed.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator Author

Annotation dispositions for head 6cbb91d:

  • Failures: there are 317 on this head and 317 on main's tip e77793f, this PR's merge base, from the same checks. They come from two sources:

    • _selftest.yml negative controls: the _ci-* fixtures, Process completed with exit code 1. on the continue-on-error steps, and the version-check contract fixture.
    • Whole-tree warn-only scans of website/reference/*.qmd.

    None comes from a line this PR changes, so they are false positives for this PR.

  • The failure lines that differ from main differ only in run-specific values: the merge-drops fixture's merge SHA, and the quarto-links fixture's orphan image name, which carries the run id. These are the same controls, so they are false positives.

  • Warnings: there are 131 here and 139 on main. All are pre-existing or selftest controls, so they are false positives:

    • yamllint comment spacing;
    • the actionlint and zizmor summaries;
    • the credential-shape fixture;
    • the localhost model endpoint;
    • the save-pr-diff failure cases;
    • the open-issue ceiling, whose count varies by run (219 on this run);
    • the antigravity retry.
  • No annotation points at a file this PR changes: annotation-gate/annotation_gate.py, its test file, or changelog.d/annotation-gate-deferral-summary.changed.md. The five AI-tell warnings on other changelog.d/ fragments come from a whole-tree scan and are on main too.

  • Notices are routine and grouped.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@d-morrison

Copy link
Copy Markdown
Collaborator Author

Cross-model review (agy, gemini-4-hydrogen-high) of head 6cbb91d:

Summary

Round 2 review of Morrison-Lab/gha#2152 at commit 6cbb91dc607370dcf6af092d4c0fccf4e243bf88.

Both findings from the earlier review are resolved:

  1. PR description vs. test diff: The PR description now accurately lists all five new tests in annotation-gate/tests/test_annotation_gate.py (test_a_deferred_gate_names_the_runs_it_waits_on_in_the_summary, test_a_deferral_notice_escapes_run_names, test_command_data_escapes_percent_and_line_breaks, test_waiting_lines_keep_a_run_name_on_one_line, and test_waiting_lines_count_minutes_since_creation), and test_a_deferred_gate_names_the_runs_it_waits_on_in_the_summary asserts both the job summary content and that the single ::notice line names the waiting run.
  2. Docstring and PR description vs. log output: Both report_deferral()'s docstring and the PR description now accurately state that the log line still reports only the bare count, while the waiting runs are named in the ::notice title=Annotation gate deferred:: command and in $GITHUB_STEP_SUMMARY.

The implementation (one_line(), waiting_lines(), command_data(), report_deferral(), and the call from gate_commit()) and the changelog fragment changelog.d/annotation-gate-deferral-summary.changed.md have no functional or documentation defects.

Findings

None.

Verdict

All round 1 findings are resolved and no new defects were found in the diff or PR description.

VERDICT: CLEAN

Reviewed-Commit: 6cbb91d

Posted by Claude Code (AI agent) --- not written by a human.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator Author

Annotation dispositions for head a035b3e:

  • Failures: 321 on this head, against 317 on the older main tip e77793f used for 6cbb91d. The difference is the _ci-ball-neighborhoods selftest fixture that gha#2158 added to main, which this head now carries. Everything else is the same set as before: _selftest.yml negative controls (the _ci-* fixtures, Process completed with exit code 1. on continue-on-error steps, the version-check contract fixture) and whole-tree warn-only scans of website/reference/*.qmd. False positive for this PR: none comes from a line it changes.
  • Lines that differ only by run-specific values (the quarto-links fixture orphan image name, which carries run id 44713012951, and the merge-drops fixture SHA) are the same controls. False positive.
  • Warnings: 131. yamllint comment spacing, actionlint and zizmor summaries, the credential-shape fixture, the localhost model endpoint, save-pr-diff failure cases, the open-issue ceiling and the ball-neighborhoods warn-mode control are pre-existing or selftest controls. False positive.
  • No annotation points at a file this PR changes (annotation-gate/annotation_gate.py, annotation-gate/tests/test_annotation_gate.py, changelog.d/annotation-gate-deferral-summary.changed.md).
  • Notices: routine, grouped.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@d-morrison

Copy link
Copy Markdown
Collaborator Author

Cross-model review (agy, gemini-4-hydrogen-high) of head a035b3e:

Summary

PR Morrison-Lab/gha#2152 at commit a035b3e4e749f5a872efa319f22a28a6e3c96c5e adds one_line(), waiting_lines(), command_data(), and report_deferral() in annotation-gate/annotation_gate.py, wires gate_commit() to call report_deferral(head_sha, waiting) while preserving the existing deferral log line and exit behavior, adds five unit tests in annotation-gate/tests/test_annotation_gate.py, and adds changelog.d/annotation-gate-deferral-summary.changed.md.

The implementation, tests, docstrings, changelog entry, and PR description are consistent and free of defects.

Findings

None.

Verdict

VERDICT: CLEAN

Reviewed-Commit: a035b3e

Posted by Claude Code (AI agent) --- not written by a human.

@github-actions

Copy link
Copy Markdown
Contributor

Claude finished review — View run


PR #2152 has no new findings at a9b7fbf. I read the diff and did not re-run the tests this round.

The only change since the last review is a9b7fbf, a merge of origin/main into the branch. Its result matches git's automatic merge of its parents, so it hand-resolved nothing. The one commit it brought in, fe96fa7 (check-template-identity, #2159), is already on main, so it is outside this PR's scope.

The PR's own diff against its base is three files: annotation-gate/annotation_gate.py, annotation-gate/tests/test_annotation_gate.py and the changelog fragment changelog.d/annotation-gate-deferral-summary.changed.md. The code in them is the same as at the last reviewed head, a035b3e:

  • one_line collapses whitespace in run names and links.
  • waiting_lines returns one physical line per waiting run.
  • command_data escapes %, CR and LF for the ::notice line.
  • report_deferral is called from gate_commit.
  • The tests cover the escaping, the one-line collapse, the age in minutes and the summary file.

The prior round on a035b3e was Ready for merge (comment). That verdict still stands. The earlier rounds ran annotation-gate/tests and got 78 passed.

The PR adds no math or notes content, so the macro and content rules do not apply. It changes no permissions or interfaces.

Verdict

Ready for merge

Structured Review Data (JSON)
{
  "schema_version": "1.1",
  "reviewer": "claude",
  "commit_sha": "a9b7fbf905d56675243f96a6141a08f7ea048f6c",
  "verdict": "CLEAN",
  "findings": [],
  "detailed_assessment": "Re-read the full PR diff: report_deferral, waiting_lines, one_line and command_data in annotation_gate.py are unchanged since the last clean round, and the new merge commit matches git's automatic merge.",
  "holistic_assessment": "The only new commit is a clean merge of main that brings in an unrelated check already on main, so scope, regression and interface risk are unchanged and the earlier clean verdict stands."
}

Reviewed commit: a9b7fbf

Reviewed commit: a9b7fbf

@github-actions

Copy link
Copy Markdown
Contributor

💰 Cost: $0.8345 (review) — run

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

annotation-gate: a gate that defers behind a queued run posts nothing, so a head reads as "gate never ran"

2 participants