Skip to content

ci(bench): revalidate the benchmarks website once per develop push - #11192

Merged
lwwmanning merged 3 commits into
developfrom
wm/bench-revalidate-once
Nov 4, 2026
Merged

lwwmanning merged 3 commits into
developfrom
wm/bench-revalidate-once

Conversation

@lwwmanning

@lwwmanning lwwmanning commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Before: each benchmark job sent POST /api/revalidate and a warm pass of about 26 requests to bench.vortex.dev. During develop runs, this occurred about 15 to 60 times an hour.

After: a reusable refresh-bench-site.yml workflow sends one revalidate and one warm pass. develop-bench.yml and nightly-bench.yml call it once per run, after all ingests, with always(). commit-metadata.yml calls it after its commit row. Thus a develop push gets two refreshes and a nightly run gets one. A failed revalidate is tried again three times. Ingestion stays in each job. The request shape agrees with bench/CONTRACT.md.

Checks: yamllint --strict, ruff and pytest pass. A local test server showed the correct requests. typos was not installed. I read the workflow graph; no workflow run was started.

🤖 Generated with Claude Code

post-ingest.py sent POST /api/revalidate and a warm pass of about 26
requests after every benchmark job, so about 15 to 60 times an hour
while develop benchmarks ran. commit-metadata.yml sent one more.

Move the revalidate and the warm pass to scripts/refresh-bench-site.py.
A new refresh-site job in develop-bench.yml runs it once, after the
bench and sql jobs finish, with always() so that a failed benchmark
does not stop the results of the other jobs from appearing. Ingestion
stays in each job. The request shape is unchanged.

Signed-off-by: Will Manning <will@willmanning.io>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T15:30:21.719018Z 2decdd3 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@lwwmanning
lwwmanning enabled auto-merge (squash) November 4, 2026 07:00

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 790d619d11

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/post-ingest.py Resolved
A workflow_dispatch run of commit-metadata.yml has no refresh-site job
after it, so it revalidates the benchmarks website itself. Push runs
still leave the revalidate to develop-bench.yml.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning

Copy link
Copy Markdown
Contributor Author

@codex review

@lwwmanning
lwwmanning disabled auto-merge November 4, 2026 15:16

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b10818d3e3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/develop-bench.yml Resolved
Comment thread .github/workflows/commit-metadata.yml Resolved
Comment thread scripts/refresh-bench-site.py Resolved
…idate

Move the refresh job into a reusable refresh-bench-site.yml workflow and
call it once per run from develop-bench.yml, nightly-bench.yml and
commit-metadata.yml. Nightly runs ingest production results too, and
commit-metadata.yml does not wait for develop-bench.yml, so each
refreshes after its own ingest.

refresh-bench-site.py now tries the revalidate POST again after 5, 15
and 45 seconds on a transport error, HTTP 429 or HTTP 5xx.

Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 2decdd3605

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@lwwmanning
lwwmanning merged commit 0e8442b into develop Nov 4, 2026
92 of 93 checks passed
@lwwmanning
lwwmanning deleted the wm/bench-revalidate-once branch November 4, 2026 15:34
lwwmanning added a commit to vortex-data/vortex.dev that referenced this pull request Nov 8, 2026
vortex-data/vortex#11192 moved the cache revalidate out of
scripts/post-ingest.py into scripts/refresh-bench-site.py, which a
reusable workflow runs once per CI run. Update the contract, the
architecture docs, the secrets runbook, and two code comments to name
the new caller and describe when it runs and how it handles failure.
No behaviour changes.

Signed-off-by: Will Manning <will@willmanning.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants