Skip to content

Release script: agent-written summary in the bump PR - #167

Merged
JacobStephens2 merged 2 commits into
mainfrom
issue-159
Sep 30, 2026
Merged

JacobStephens2 merged 2 commits into
mainfrom
issue-159

Conversation

@JacobStephens2

Copy link
Copy Markdown
Owner

Summary

The release script now has claude write the bump PR's summary. If the agent fails, it falls back to GitHub's generated notes.

 pr_body <base> <tag>
-  notes = gh generate-notes
+  diff  = version_diff <base>
+  release_summary <base> <tag> <diff>
+    input = <diff> + "### #N: title\n\nbody" for each PR merged since the last v* tag
+    prompt file + input | claude -p --tools '' --strict-mcp-config
+    if it failed or printed nothing
+      warn on stderr
+      "_The agent summary was unavailable, so this is GitHub's generated notes._"
+      gh generate-notes
   ## Summary  <!-- release-summary:start --> … <!-- release-summary:end -->
   ## Version diff
 scripts/
 ├── release.sh
+└── release-summary.md   # the agent's prompt, with the emphasis rules and the 0.3.0 summary as the model
 tests/fakes/
 ├── claude               # + plain print mode: records stdin, prints only the script's output
 └── gh                   # + `pr view <number>` and `--jq .<field>`
  • The PRs merged since the last tag come from the Merge pull request #N merge commits in <last tag>..origin/main. The bump PR isn't merged yet when this runs, so it is never among them.
  • The README's Releasing section describes the agent summary and the fallback.

Evidence

  • Before: on main, the summary section holds GitHub's generated notes and claude is never called.
    After: cargo test --test release passes all 9 tests, including 4 new ones:
the_agent_gets_the_prompt_file_then_the_version_diff_and_the_prs_merged_since_the_last_tag
  stdin == prompt file + version diff + "#2: Add the frobnicator" + body + "#3: Tidy the site"
  (the PR merged before v0.1.0 and the bump PR are left out)
the_agent_runs_in_print_mode_with_tools_disabled         -p, --tools '', --strict-mcp-config
the_pr_body_has_the_agents_summary_in_a_marked_section_then_the_version_diff
when_the_agent_fails_or_prints_nothing_the_summary_is_the_generated_notes
  exit 1 / blank output → a stderr warning, the note, then the generated notes; v0.2.0 is still tagged
  • The full cargo test --no-fail-fast passes. In one earlier run, merge_run's ctrl_c_after_the_merge_… hit its 10-second "agent never started" deadline while the machine was under heavy load. It passed on the re-run, and it doesn't touch the release script.
  • shellcheck scripts/release.sh and cargo clippy --all-targets -- -D warnings are clean.

Merge Danger

Door: two-way

This only changes the maintainer's release script and its tests. Nothing ships in the binary. The tag push it leads to was already one-way.

Blast Radius: release notes

At worst, a poor agent summary ends up in a bump PR body, and the maintainer can edit it. Releasing now needs claude on PATH. If claude is missing or fails, the script falls back to the generated notes rather than stopping.

Unaddressed findings

Standards

  • Two gh pr view calls per PR (title, then body): a single call would need a jq expression that joins a multi-line body. Separate calls stay simple, and a release has only a few PRs.
  • The fake gh builds a second field map for --jq: pr_fields deliberately leaves out title and body, which pr list doesn't return, so the --jq path adds them locally.
  • The argv test checks exact flags: "tools disabled" is a safety property the spec names, so asserting the flags is the point of the test.

Spec

  • The prompt says the summary is also for the Release page: that is where Cut a release with one command: bump PR, agent-written summary, merge and tag #150 puts the summary. The wording tells the agent who reads it and doesn't implement the Release page hook.
  • --strict-mcp-config beyond "tools disabled": --tools '' only disables built-in tools, so this flag also keeps MCP tools out, as Cut a release with one command: bump PR, agent-written summary, merge and tag #150 intends ("it can't take actions").
  • Squash-merged or rebase-merged PRs are missed: this repo merges with merge commits, and the release script itself merges that way.
  • A failure to collect the input stops the release instead of falling back: the spec's fallback covers agent failures. A gh failure while reading PRs also means the generated notes would likely fail.
  • An untested assumption that "not logged in" exits non-zero: I couldn't check the real logged-out claude in this headless session. The whitespace-only check also catches an empty result.
  • No test with a previous bump PR inside the range: that bump's merge commit carries the previous tag, so it sits outside <last tag>..main by construction.

Closes #159

🤖 Generated with Claude Code

JacobStephens2 and others added 2 commits September 30, 2026 02:36
The release script pipes the prompt in scripts/release-summary.md, the
version diff and each PR merged since the last tag to claude -p with no
tools, and puts its output in the summary section. If claude fails or
prints nothing, it warns and falls back to GitHub's generated notes with
a note saying so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Write the version diff section once, have release_summary find the
previous tag itself, and drop the prompt's bullet rule, which pulled
against the 0.3.0 prose it holds up as the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JacobStephens2
JacobStephens2 merged commit 814d491 into main Sep 30, 2026
13 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-159 branch September 30, 2026 02:43
JacobStephens2 added a commit that referenced this pull request Sep 30, 2026
Brings in the claude-written release summary (#167). The release tests
keep both sides' helpers: the CI-run and origin-refs helpers for the
refusals, and the fake claude helpers for the summary. The CI refusal
test's merge_pr call takes the body argument main added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JacobStephens2 added a commit that referenced this pull request Sep 30, 2026
Brings in #167's agent-written release summary. The PR body is now built
in open_pr from the bump commit's sha rather than HEAD, since the script no
longer runs inside the worktree, so version_diff takes <base> <head>. The
fake gh's pr view keeps both --jq support and headRefOid through pr_fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JacobStephens2 JacobStephens2 mentioned this pull request Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release script: agent-written summary in the bump PR

1 participant