Release script: agent-written summary in the bump PR

README.md

@@ -334,13 +334,13 @@ Running the test suite (`cargo test`) also needs **`python3`** on `PATH`: the in
 
 ## Releasing
 
-From a clone of this repo, signed in to `gh`, run the release script with the new version:
+From a clone of this repo, signed in to `gh` and with `claude` logged in, run the release script with the new version:
 
 ```sh
 scripts/release.sh 0.4.0
 ```
 
-It works from `origin/main` in a temporary worktree, so your checked-out branch and uncommitted changes don't matter and aren't touched. It opens a pull request from `release-<version>` that bumps the version in `Cargo.toml` and `Cargo.lock` and nothing else, with GitHub's generated notes for the pull requests merged since the last tag and the version diff as its body. It waits for the pull request's checks, merges it with a merge commit, then tags that merge commit `v<version>` and pushes the tag. It prints a line on stderr for each step. If the checks fail, it stops before merging and leaves the pull request open.
+It works from `origin/main` in a temporary worktree, so your checked-out branch and uncommitted changes don't matter and aren't touched. It opens a pull request from `release-<version>` that bumps the version in `Cargo.toml` and `Cargo.lock` and nothing else. Its body is a summary of the release, then the version diff. `claude -p`, with no tools, writes the summary from the version diff and the title, number and body of each pull request merged since the last tag, following the prompt in [`scripts/release-summary.md`](scripts/release-summary.md). If `claude` fails or prints nothing, the script warns and uses GitHub's generated notes as the summary instead, with a note saying so. It waits for the pull request's checks, merges it with a merge commit, then tags that merge commit `v<version>` and pushes the tag. It prints a line on stderr for each step. If the checks fail, it stops before merging and leaves the pull request open.
 
 The tag starts the release workflow, generated by [`dist`](https://github.com/axodotdev/cargo-dist) from `dist-workspace.toml`. It builds the Linux and macOS binaries and the installer, checks that the Linux binary starts on Rocky Linux 9 and in WSL2 with Ubuntu 24.04, and only then publishes the GitHub Release. It then publishes the same version to crates.io, skipping it if it is already there, and finally replaces the Release's body with notes generated from the merged pull requests.
 

scripts/release-summary.md

@@ -0,0 +1,15 @@
+Write the summary of a thirdshift release for its release pull request and its GitHub Release page. Below this prompt are the release's version diff and the title, number and body of each pull request merged since the last release.
+
+Print only the summary, in GitHub Markdown, with no heading and nothing before or after it:
+
+1. A short headline paragraph on what the release is about: its biggest change for someone running `thirdshift`, in a sentence or two.
+2. What changes for someone running `thirdshift`, first: new commands, then behaviour that is new or changed, most important first.
+3. Site, docs and CI work in one short grouped line after that, such as "Also: site, docs and CI updates (#12, #14, #15)." Leave it out if there was none.
+
+Reference every change by its pull request number, as `#<number>`. Several pull requests that make one change can share a bullet.
+
+Claim nothing the pull request titles and bodies don't say. Don't guess at motives, impact or details they leave out. When a title is all there is, say no more than the title does.
+
+Aim for the tone and density of the 0.3.0 summary:
+
+> 0.3.0 ships the changes merged since v0.2.0. The headline is the **Merge run** (`thirdshift merge <Issue URL>`): #89 (define Merge run, Self-merge and Foreign commit), #96 (tracer bullet), #97 (a failed Self-merge goes back round the Repair loop; policy refusals leave the PR ready), #98 (post-merge steps: delete the Issue branch, close the issue), #99 (review Foreign commits with a Repair). Also #95 (Repair and base-move budgets raised to 5) and the site work in #72–#87.

scripts/release.sh

@@ -6,16 +6,23 @@
 #
 # Usage: scripts/release.sh <version>
 #
-# Run it in a clone of this repo, signed in to gh. It works from origin/main in
-# a temporary worktree, so the branch checked out where it runs and any
-# uncommitted changes there are neither used nor changed.
+# Run it in a clone of this repo, signed in to gh and with claude logged in. It
+# works from origin/main in a temporary worktree, so the branch checked out
+# where it runs and any uncommitted changes there are neither used nor changed.
+#
+# claude, with no tools, writes the PR's summary from the prompt in
+# release-summary.md beside this script. If it fails, the summary is GitHub's
+# generated notes instead, and the script warns and carries on.
 #
 # Prints a progress line on stderr for each step. Exits 0 once the tag is
 # pushed; if the PR's checks fail, it leaves the PR open and exits 1.
 set -euo pipefail
 # So a failure inside $(…) stops the script too.
 shopt -s inherit_errexit
 
+# The prompt the summary agent gets ahead of the release's input.
+summary_prompt=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/release-summary.md
+
 # Seconds between reads of the PR's workflow runs.
 poll_seconds=${RELEASE_POLL_SECONDS:-15}
 
@@ -42,6 +49,7 @@ main() {
 	head=$(git rev-parse HEAD)
 	git push --quiet origin "HEAD:refs/heads/$branch"
 
+	progress "writing the summary with claude"
 	# Built first so a failure to build it stops the script, as it would not
 	# inside the gh command line.
 	body=$(pr_body "$base" "$tag")
@@ -102,21 +110,26 @@ rewrite() {
 # pr_body <base> <tag>
 # The body of the bump PR on HEAD, which <tag> will name, off <base>: the
 # summary, between markers the Release page step can find, then the version
-# diff. For now the summary is GitHub's generated notes for the PRs merged
-# since the last tag.
+# diff.
 pr_body() {
-	local previous notes
-	previous=$(git describe --tags --abbrev=0 --match 'v[0-9]*' "$1" 2>/dev/null || true)
-	notes=$(gh api 'repos/{owner}/{repo}/releases/generate-notes' \
-		-f tag_name="$2" -f target_commitish="$1" \
-		${previous:+-f previous_tag_name="$previous"} --jq .body)
+	local diff summary
+	diff=$(version_diff "$1")
+	summary=$(release_summary "$1" "$2" "$diff")
 	cat <<-EOF
 		## Summary
 
 		<!-- release-summary:start -->
-		$notes
+		$summary
 		<!-- release-summary:end -->
 
+		$diff
+	EOF
+}
+
+# version_diff <base>
+# The version diff section: HEAD's changes from <base>.
+version_diff() {
+	cat <<-EOF
 		## Version diff
 
 		\`\`\`diff
@@ -125,6 +138,48 @@ pr_body() {
 	EOF
 }
 
+# release_summary <base> <tag> <version diff section>
+# The agent's summary of the release, or, if the agent fails or prints
+# nothing, a note saying so and GitHub's generated notes.
+release_summary() {
+	local previous input summary
+	previous=$(git describe --tags --abbrev=0 --match 'v[0-9]*' "$1" 2>/dev/null || true)
+	input=$(summary_input "$1" "$previous" "$3")
+	# In an if, so the agent failing falls back rather than stopping the script.
+	if summary=$({ cat "$summary_prompt" && echo && echo "$input"; } |
+		claude -p --tools '' --strict-mcp-config) &&
+		[ -n "${summary//[[:space:]]/}" ]; then
+		echo "$summary"
+		return
+	fi
+	progress "warning: the agent summary is unavailable, so the summary is GitHub's generated notes"
+	echo "_The agent summary was unavailable, so this is GitHub's generated notes._"
+	echo
+	gh api 'repos/{owner}/{repo}/releases/generate-notes' \
+		-f tag_name="$2" -f target_commitish="$1" \
+		${previous:+-f previous_tag_name="$previous"} --jq .body
+}
+
+# summary_input <base> <previous tag> <version diff section>
+# What the agent summarises: the version diff, then the title, number and body
+# of each PR merged into <base> since <previous tag>, oldest first. The bump PR
+# isn't merged yet, so it isn't among them.
+summary_input() {
+	local number title body
+	cat <<-EOF
+		$3
+
+		## Pull requests merged since ${2:-the first commit}
+	EOF
+	git log --reverse --merges --format=%s "${2:+$2..}$1" |
+		sed -n 's/^Merge pull request #\([0-9][0-9]*\) from .*/\1/p' |
+		while read -r number; do
+			title=$(gh pr view "$number" --json title --jq .title)
+			body=$(gh pr view "$number" --json body --jq .body)
+			printf '\n### #%s: %s\n\n%s\n' "$number" "$title" "$body"
+		done
+}
+
 # wait_for_workflow_runs <sha>
 # Waits until <sha> has workflow runs and all of them have completed, then
 # prints the names of those that did not succeed, comma-separated, or nothing.

tests/fakes/claude

@@ -21,6 +21,10 @@ The `init` line carries session id `fake-session-<n>` for the n-th call. Lines
 the script appends to the file named by $FAKE_CLAUDE_AFTER_RESULT are emitted
 after the closing `result` line, e.g. to kill a background task as the session
 ends.
+
+Called without --output-format, as `claude -p` with its prompt on stdin, it
+records stdin as `stdin` and prints only what the script prints, as print
+mode's plain text output does.
 """
 
 import json
@@ -74,6 +78,7 @@ def main():
         == 0
     )
 
+    text_mode = "--output-format" not in argv
     record_path = os.environ["FAKE_CLAUDE_RECORD"]
     records = []
     if os.path.exists(record_path):
@@ -88,19 +93,23 @@ def main():
             "merging": merging,
             "plugin_dir": plugin_dir,
             "plugin_files": plugin_snapshot(plugin_dir) if plugin_dir else {},
+            "stdin": sys.stdin.read() if text_mode else None,
         }
     )
     with open(record_path, "w") as f:
         json.dump(records, f, indent=2)
 
+    script = script_for(records)
+    if text_mode:
+        sys.exit(subprocess.run(["bash", "-e", script]).returncode)
+
     init = {
         "type": "system",
         "subtype": "init",
         "cwd": os.getcwd(),
         "session_id": f"fake-session-{len(records)}",
     }
     print(json.dumps(init), flush=True)
-    script = script_for(records)
     # Beside the record, not in $TMPDIR, which tests expect to be left empty.
     after_result = f"{record_path}.after-result.{len(records)}"
     env = dict(os.environ, FAKE_CLAUDE_AFTER_RESULT=after_result)

tests/fakes/gh

@@ -159,15 +159,27 @@ def pr_create(state, flags):
 
 
 def pr_view(state, positional, flags):
+    """By head branch, its newest PR, or by number. `--jq .<field>` prints
+    that one field raw, as gh does for a string."""
     head = positional[0]
-    matches = [pr for pr in state["prs"] if pr["head"] == head]
+    if head.isdigit():
+        matches = [pr for pr in state["prs"] if pr["number"] == int(head)]
+    else:
+        matches = [pr for pr in state["prs"] if pr["head"] == head]
     if not matches:
         die(f'no pull requests found for branch "{head}"')
     pr = matches[-1]
     wanted = flags["json"].split(",")
+    jq = flags.get("jq")
+    if jq is not None:
+        if not jq.startswith(".") or jq[1:] not in wanted:
+            die(f"fake gh: unsupported --jq {jq}", 2)
+        fields = dict(pr_fields(pr), title=pr["title"], body=pr.get("body", ""))
+        print(fields[jq[1:]])
+        return
     fields = pr_fields(pr)
     if "headRefOid" in wanted:
-        fields["headRefOid"] = pr.get("headRefOid") or origin("rev-parse", f"refs/heads/{head}")
+        fields["headRefOid"] = pr.get("headRefOid") or origin("rev-parse", f"refs/heads/{pr['head']}")
     fields["closingIssuesReferences"] = [
         {"number": int(number)} for number in CLOSING_KEYWORD.findall(pr.get("body", ""))
     ]