Skip to content

Require an AI cross-review before merge - #229

Open
lwwmanning wants to merge 10 commits into
mainfrom
wm/vortex-cross-ai-review-f2ab62
Open

lwwmanning wants to merge 10 commits into
mainfrom
wm/vortex-cross-ai-review-f2ab62

Conversation

@lwwmanning

Copy link
Copy Markdown
Contributor

What changes

This adds a merge gate for AI review to both sites' pull requests, in a new workflow, ai-review.yml. The other checks stay as they are.

Before, Codex reviewed some pull requests on its own, and Claude reviewed only when someone tagged @claude. Neither review blocked a merge. After, the gate sets the commit status AI cross-review on the head commit. Every pull request needs a clean Codex review of that commit. If the footer or the author names Codex, the pull request also needs a clean Claude review, which the workflow runs.

A P0 or P1 finding fails the gate, so auto-fix can act on it. When Codex has not reviewed the head commit, the gate posts @codex review. Renovate and Dependabot pull requests pass without review.

After merge, the main ruleset must require AI cross-review before the gate blocks anything.

How it was checked

bun test scripts/ai-review-gate.test.ts, the typecheck and bun run check:ci pass. Dry runs against #216, #220, #225, #206 and #227 gave the expected status for each.

  • Every commit is signed off (git commit -s), for the DCO check.
  • bun run check:ci passes.
  • Each new image or other asset has an annotation in REUSE.toml.

🤖 Generated with Claude Code

Every pull request needs a Codex review of its head commit. A pull request
whose footer or author names Codex also needs a Claude review. The gate sets
the commit status "AI cross-review" for the main ruleset to require.

Signed-off-by: Will Manning <will@willmanning.io>
@vercel

vercel Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
vortex.dev Ready Ready Preview Nov 21, 2026 2:38am UTC

Request Review

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

Comment thread .github/workflows/ai-review.yml Fixed
Comment thread .github/workflows/ai-review.yml Fixed
…view-f2ab62

Signed-off-by: Will Manning <will@willmanning.io>

# Conflicts:
#	CLAUDE.md
@lwwmanning
lwwmanning marked this pull request as ready for review November 21, 2026 01:51
CodeQL flagged the explicit checkout of the head commit as an untrusted
checkout, because the workflow also runs on issue_comment.

Signed-off-by: Will Manning <will@willmanning.io>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: 🔄 Running

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-11-21T02:38:18.504146Z d46d93d New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

CodeQL flags any job that runs the pull request's code in a workflow that
also runs on issue_comment. ai-review.yml now runs on pull_request only,
and ai-review-recheck.yml reruns main's copy of the gate on comments and
reviews. The gate's token also gets pull-requests: write, which a comment
on a pull request needs.

Signed-off-by: Will Manning <will@willmanning.io>
@github-actions

Copy link
Copy Markdown

@codex review

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🔦 Lighthouse audit

URL Perf A11y Best practices SEO
/ 🟢 100 🟢 100 🟢 100 🟢 100
/bench 🟢 100 🟢 100 🟢 100 🟢 100
/bench/chart/ct.eyJrIjoiQ29tcHJlc3Npb25UaW1lIiwiZGF0YXNldCI6ImFyYWRlIiwiZGF0YXNldF92YXJpYW50IjpudWxsfQ 🟢 100 🟢 100 🟢 100 🟢 100
/blog 🟢 99 🟢 100 🟢 100 🟢 100
/blog/btrblocks-compressor 🟢 100 🟢 100 🟢 100 🟢 100

Run 44856970609 · 2026-11-21 02:39 UTC

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e51dd80774

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ai-review.yml Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread .github/workflows/ai-review.yml Resolved
Comment thread .github/workflows/ai-review.yml Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread .github/workflows/ai-review.yml Resolved
Codex's review found that the gate ran in the pull request's context: forks
and Dependabot PRs got a read-only token, a PR could change the script that
judged it, and fork PRs could never get a Claude review. The workflow now
runs on pull_request_target, comments and reviews, and every job checks out
main. Claude reads the PR through the API.

The gate also asks Codex again after 20 minutes without a start (three times
at most), retries a Claude review that ended without a verdict, and ignores
dismissed Codex reviews. ci.yml runs the gate's tests.

Signed-off-by: Will Manning <will@willmanning.io>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 25fbcceb11

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
The gate sets itself to pending before it reads the reviews, so a failed
run cannot leave an old success in place. It fails when another open PR
has the same head commit, because a commit status counts for every PR on
that commit. A submitted Codex review of the head now counts even while
the summary comment still says the review is running.

Signed-off-by: Will Manning <will@willmanning.io>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 20e6373921

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
The Claude status name now holds the PR number, so a review of one PR does
not count for another PR on the same commit. The gate counts Claude's
blocking findings from its unresolved P0 and P1 threads, as it does for
Codex, so resolving a thread and commenting clears it without a new push.

Signed-off-by: Will Manning <will@willmanning.io>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5093392b2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread scripts/ai-review-gate.ts Resolved
A PR that moves to another base branch keeps its head commit, but its diff
can change, so the gate posts a marker and fails until a new commit
arrives. Only P0 and P1 threads count as evidence for Claude's counted
findings. The gate reads every page of the commit's statuses, and counts
only marker comments from github-actions[bot].

Signed-off-by: Will Manning <will@willmanning.io>
…view-f2ab62

Signed-off-by: Will Manning <will@willmanning.io>

# Conflicts:
#	CLAUDE.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3516cffac1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ai-review-gate.ts Resolved
Comment thread .github/workflows/ai-review.yml Resolved
Comment thread scripts/ai-review-gate.ts Resolved
Comment thread .github/workflows/ai-review.yml Resolved
Comment thread .github/workflows/ai-review.yml Resolved
Drop the pull_request_review trigger, which runs the PR's copy of the
workflow. Claude reviews only PRs by authors with write access; an
outside PR that Codex helped write needs a maintainer's approval of the
head commit. Record a base change as a commit status before the notice
comment. Mark a Claude review as started only after setup, and retry a
review that stays pending for 45 minutes. Check at verdict time that
each blocker Claude counts has an open thread.

Signed-off-by: Will Manning <will@willmanning.io>

This branch was successfully deployed

1 active deployment
Preview — d46d93d6 Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants