Skip to content

production serves a product page for a provider none of its benches list - #2981

Open
Flotapponnier wants to merge 1 commit into
mainfrom
fix/confine-provider-index-main
Open

Flotapponnier wants to merge 1 commit into
mainfrom
fix/confine-provider-index-main

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

/products/validationcloud answers 200 on openchainbench.com right now,
with a title, a description and links to six keyed RPC benches. Open any of
those six on production and the private cohort lists five providers, none of
them Validation Cloud:

prod /api/bench/ethereum-rpc/variant?tier=keyed
  -> 5 rows: alchemy, chainstack, getblock, quicknode, dwellir
prod /products/validationcloud
  -> 200, "Validation Cloud: RPC benchmark 2026, live rank"

No spec on main declares the provider: git grep validationcloud -- benchmarks/
on this branch returns nothing.

Why

The provider index is built from the materialize worker's bench blobs, and the
worker clones dev at Docker build time while production runs main. The blob
therefore credits appearances this deployment's own specs do not declare, and
the index ends up believing in a provider that cannot appear in a single table
here. It is absent from the sitemap, which is the only reason it has not been
crawled, but it is reachable and carries no noindex.

This is not specific to one provider. Any provider added to a bench on dev
gets a production product page as soon as the worker rebuilds, for as long as
the change sits on dev.

The fix

Confine the index to what this deployment declares: for each bench, if the
local spec enumerates providers, drop blob appearances for any provider the
spec does not name. Benches whose spec declares no provider list are untouched,
so nothing outside the keyed RPC cluster changes behaviour.

This is the guard half of #2978, taken alone. That PR also adds the
provider to the keyed probe's harness config, which would start measuring
Validation Cloud from main. This carries only the guard, so production stops
publishing a page for a provider it does not measure and gains nothing about
that provider. #2978 can stay open or be closed independently.

The test reads the repository's real specs rather than fixtures, because the
defect is a disagreement between a spec and a blob: a synthetic spec would
prove the filter filters, not that the filter matches what we ship.

Gates

pnpm exec tsc --noEmit    clean
bun test src/             479 pass, 0 fail (4 new)

After merge

Production redeploys from main automatically. The product page should start
answering 404; the ISR entry for it may serve the 200 until it revalidates.
Worth re-checking the URL once the deploy lands.

One thing to know before merging: pushing to main redeploys the three
Railway probe services, which build from main. main has no
validationcloud in the keyed harness's provider list, so the Validation
Cloud measurement stops on the next push to main
, whether or not it is this
one. That is pre-existing and is the trade #2978 exists to resolve; this PR
neither causes nor avoids it.

/products/validationcloud answers 200 on openchainbench.com right now, with a
title, a description and links to six keyed RPC benches. Open any of those six
on production and the private cohort lists five providers, none of them
Validation Cloud. No spec on main declares it; `git grep validationcloud
benchmarks/` on this branch returns nothing.

The provider index is built from the materialize worker's bench blobs, and the
worker clones `dev` at Docker build time while production runs `main`. So the
blob credits appearances that this deployment's own specs do not declare, and
the index believes in a provider that cannot appear in a single table here. It
is not in the sitemap, which is the only reason it has not been crawled, but it
is reachable and carries no noindex.

The fix is to confine the index to what this deployment declares: for each
bench, if the local spec enumerates providers, drop blob appearances for any
provider the spec does not name. Benches whose spec declares no provider list
are untouched, so nothing outside the keyed RPC cluster changes behaviour.

This is the guard half of #2978, taken alone. That PR also adds the provider to
the keyed probe's harness config, which would start measuring Validation Cloud
from main; this carries only the guard, so production stops publishing a page
for a provider it does not measure and gains nothing about that provider.

The test reads the repository's real specs rather than fixtures, because the
defect is a disagreement between a spec and a blob: a synthetic spec would
prove the filter filters, not that the filter matches what we actually ship.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant