Repository navigation
start the Validation Cloud probe, and keep it off production - #2978
Flotapponnier wants to merge 2 commits into
Conversation
… them One file, seven lines: validationcloud joins the provider list and gets a budget. Nothing else from dev comes with it. The probe services build from main, so until this lands they run a binary whose provider list has no validationcloud in it, and a cell is only probed when both its env var AND its provider exist. The eighteen endpoint variables have been on rpc-keyed-us, rpc-keyed-eu and rpc-keyed-sgp since yesterday, doing nothing. This changes nothing a reader sees on production: the six RPC specs that declare the provider are on dev and render on staging only. What it changes is that samples start flowing, so those staging rows stop being empty. The budget is a runaway guard at 600k requests per region per month, not a quota. Six chains at the 120 s cadence cost 130k. It wants replacing with the real figure once the account's plan is disclosed, which is also what the row's "plan not yet disclosed" tag is waiting on.
The providers blob is built by the worker from the dev checkout and read by both deployments, so production can be handed a profile that only makes sense on staging. confineToThisDeploy already dropped appearances on benches that do not exist here, which covers a dev-only bench. It did not cover the other half: a bench that exists on BOTH branches while only dev's copy declares a given provider. That is the normal state while a provider is trialled on staging, and it is the state right now. The six RPC specs that declare Validation Cloud are on dev; main's copies of those same benches do not declare it. As soon as the probe starts and the worker republishes, the blob carries a validationcloud profile whose appearances all point at benches main does serve, so the filter passed it and /products/validationcloud would have rendered on production, crediting six benchmark tables that do not list it. The bench pages were never at risk: spec.ts prunes blob results to the providers the local YAML declares. This is the same rule, applied to the product index, which is the only surface that reads the blob's provider list directly. Four tests against the repo's real specs. They also fail the day someone adds the provider to main's YAML, which is the moment the page becomes correct.
|
Closing: the premise no longer holds. Validation Cloud was removed from the keyed cohort entirely on 2026-10-18 The harness log gave the cause, identically on every chain and region: Starting the probe is not the open question any more; getting a key that works Validation Cloud remains live and correct on |
Two commits, both needed on
main, and together they do exactly one thing: start measuring, without anything appearing on production.1. The probes ignore the endpoints already set on them
One file, seven lines:
validationcloudjoins the provider list in the keyed probe harness and gets a budget.The three probe services build from
main. Until this lands they run a binary with novalidationcloudin its provider list, and a cell is probed only when both its env var and its provider exist. The eighteen endpoint variables have been sitting onrpc-keyed-us,rpc-keyed-euandrpc-keyed-sgpdoing nothing. Prometheus confirms it: five keyed providers reporting series, Validation Cloud absent.2. A provider only dev declares was about to get a production product page
Found while checking the first commit was safe to merge.
The providers blob is built by the worker from the dev checkout and read by both deployments.
confineToThisDeployalready dropped appearances on benches that do not exist here, which covers a dev-only bench. It did not cover the other half: a bench that exists on both branches while only dev's copy declares the provider. That is precisely the state we are in.So the moment commit 1 starts the probe and the worker republishes, the blob would carry a
validationcloudprofile whose appearances all point at benchesmaindoes serve. The filter would pass it, and/products/validationcloudwould have rendered on production crediting six benchmark tables that do not list it.Bench pages were never at risk:
spec.tsalready prunes blob results to the providers the local YAML declares, which is why the six RPC pages on production stay clean. This commit applies the same rule to the product index, the one surface that reads the blob's provider list directly.Four tests, run against the repo's real specs. They also fail the day someone adds the provider to main's YAML, which is the moment the page becomes correct.
What production looks like after this merges
Unchanged. No page, no spec, no copy is touched by either commit, and the new guard only removes something that should not have been there. The six specs that declare Validation Cloud are on
dev, so the rows appear on staging only.What starts happening
The probes rebuild and sample every 120 s from each region. The staging rows fill within minutes; the 24 hour median needs a day to be a real median.
Gates
pnpm typecheck,pnpm test479 pass (4 new),pnpm lintclean on the touched files,go build ./... && go vet ./...in the harness module.