Repository navigation
rpc-cost: a dedicated node priced every chain tab, including ones it does not serve - #2741
Merged
Merged
Conversation
…ed almost nothing injectLabels treats a selector pinned to `="all"` as the pooled form and replaces it when a reader picks a tab; a selector with no such matcher spans every combination instead. The queries here carried neither, so the default view asked for one provider across five profiles, three volumes and seven plan tiers at once, and the providers whose cheapest plan differs per slice fell out of the table entirely. Pooling a bill across workloads would be meaningless, averaging a $24 plan with a $10,500 one, so the harness publishes the headline slice under `kind="all"` / `bucket="all"` instead: a dapp method mix at a hundred million requests a month. The dimension labels name that slice rather than claiming to be an average.
Seven rows showed $0 on the rpc-cost leaderboard: Infura, whose daily allowance hard-stops well below the workload, and six providers with no product on the chain being priced. $0 says "free" where the truth is "unavailable", which is the one mistake this bench exists to avoid. The spec now carries a `ranked` query wired to rpc_cost_eligible, so those rows move to the unranked block with their reason instead of posting a price nobody offers. The selector labels were named after the dimension KEY, so the page offered "Venue", "Trade size" and "Kind" for what are really plan tiers, monthly request volume and workload profile. `dimension_labels` lets a spec name its own axes; every bench that sets nothing keeps the label it has today. Default view moves to 10M requests a month, the volume most readers arrive with, and the pinned `venue="all"` now carries the cheapest PAID plan: at 10M several free tiers are eligible and would otherwise take every top row, hiding the comparison. The Free tier tab answers that question on its own axis. Also: product-page entries for the seven providers that had none, and logos for four of them. BlockPI, Coinbase CDP and Instanodes publish only an .ico or a JS-rendered page, so they keep the initials fallback.
…em not Eight agents re-checked the bench against primary sources, the code, and a third independent recomputation. The money holds: 1,155 of 1,194 cells agreed first try, no cell is non-monotonic, and the winner table reproduces dollar for dollar. What did not hold was the prose, and six model bugs. The worst bug: Chainstack's archive rule was recorded and never applied. `block_age` was treated as a no-op on the theory it was already in the weights; it was not, so the indexer profile billed 1 Request Unit per request instead of 2 and the page published $3,990 for a bill of $8,990. An unknown archive rule now fails closed instead of pricing archive at par. Also fixed: the throughput gate read the billing unit when Alchemy meters the rate limit in a different one (25x apart on debug_traceTransaction), so the check now uses the throughput table and reproduces Alchemy's own 61,728 and 234,568 CU/s exactly; the dedicated cohort had no weights and rendered nine empty rows; the `cohort` label did not match the `tier` dimension key, so that selector matched nothing; reference rows were filtered out by the paid-only default; and nothing was ever Reset, so a provider that stopped being eligible kept exporting its last price. Nineteen published claims were false or unsupportable and are corrected or withdrawn, the largest being "71x" (7.6x), "five winners" (six), "the lowest per-request price in the market" (three providers beat it), "the page names the reason" (it does not, yet), and four separate promises that freshness was being checked when no baseline hash had ever been recorded. Google Blockchain Node Engine was published as live capacity; Google shuts it down on 15 December 2026. And a product page described InstantNodes, which is dead, under the slug of Instanodes, which is trading.
The audit found four things the page claimed and the code did not do. The last commit made the prose honest by dropping the claims; this one does the work instead, for three of the four. Drift detection is armed. 21 artifacts now carry a baseline hash recorded from their live body, so a provider editing its pricing surfaces as drift rather than as a stale figure that still reads as current. The remaining artifacts stay on manual re-reading on purpose: Chainstack's pricing JSON regenerates every fifteen minutes and the big marketing pages churn on any unrelated edit, so hashing them would alarm every cycle until everyone learned to ignore the alarm. Age is published either way, so a figure nobody has re-read is visible as one. An artifact we deliberately never fetch no longer reports ok=0. That gauge read as "the provider's page is down" for 38 artifacts nobody had asked about, which is a different and untrue claim; it is now simply absent. The free-tier allowance, the effective cost per million, the billing units per request and the dedicated cohort's break-even volume were all being emitted and rendered nowhere. Each now has a panel. The ledger's p50/p90/p99 columns are relabelled: the bill is a constant read from a committed catalogue, so three percentiles of it are the same number three times, and the success-rate columns are off because this bench sends no RPC traffic and had no business showing 100%. Still not done: the reason a provider is absent from a cell reaches the harness log and not the page. The prose says so rather than implying otherwise.
… pruned as missing Two bugs in shared code, both found by auditing what the page rendered rather than what the spec said. The variant fetch never sent the bucket filter, and the all-selection test that short-circuits it never considered the bucket either. So picking only a volume fetched nothing, any other fetch came back with the default volume's figures, and the result was cached under a key that DOES include the volume. Every bucket tab rendered the default one. This also broke bench 268's trade-size tabs, which is the dimension the key was added for. The ledger and the summary strip prune all-zero rows because on a latency bench a provider emitting 0 ms is dead, not instant. On a cost bench $0 is the answer: dRPC and OnFinality genuinely serve a ten-million-request month for nothing, and pruning them made the page name a paid provider as the cheapest option. Rather than change that rule for 240 benches, a spec can now declare `zero_is_a_value`, defaulting to false so nothing else moves.
…lice # Conflicts: # benchmarks/rpc-cost.yml
…ot serve The audit caught two blockers, both introduced this afternoon while fixing something else. Making the dedicated cohort unmetered so it would stop rendering nine empty rows also skipped unitsPerRequest, which held the only chain check. So AWS, whose three plans are Ethereum nodes, led the Solana tab at $97.82; Allnodes showed $320 for its Ethereum plan where its Solana plan is about $1,920; and Helius's Solana node carried an Ethereum bill. Every plan now declares the chains it serves, taken from its own published name rather than inferred from its id, and the catalogue refuses to load dedicated capacity that declares none. That hole cannot reopen quietly. The free-tier panel ranked dRPC first on the trace tab at 10.5M requests while its own free plan is archive: false, trace: false and the ledger directly below refused it. The panel now applies the same eligibility gates as the table, so an allowance a plan cannot spend on a workload is not published as one. Fixing the first surfaced a third: the break-even denominator was the cheapest simple-read rate, and simple-read is an Ethereum profile, so every Solana node was dropped for want of a denominator. The floor is now computed per chain from that chain's own baseline workload, and a break-even is labelled with the chain it was actually computed against.
…lice # Conflicts: # harnesses/rpc-cost/cmd/script/model.go # harnesses/rpc-cost/cmd/script/profiles.go # harnesses/rpc-cost/cmd/script/refresh.go # public/logos/getblock.png # src/data/provider-registry.ts # src/lib/spec-schema.ts # src/types/benchmark.ts
…s moved Four published claims were hand-written about values the model computes, so correcting Chainstack's archive rule, the throughput meter and the dedicated cohort's chain gate silently falsified them. The audit caught it; the text had been corrected once already and had rotted again by the time the model settled. Recomputed from the harness's own 1,485 series: five providers win a cell outright rather than six, and two more share a cell on a tie; the trace profile at a hundred million spans 6.7x from NOWNodes at $567.75, not 7.6x from Chainstack at $499; Alchemy and QuickNode run 1.7x to 10.6x above the cell leader, not 2.5x; and Chainstack, which the copy still named as a trace winner, wins no cell outright. The durable half of the fix is a per-bench plugin in the audit skill that recomputes all four from Prometheus and fails when the prose disagrees, plus a check that understands `dimensions.tier` partitions the cohort so the generic row count stops reporting 20 of 31 as a fault.
The SEO audit found the page handing answer engines the exact claim its own intro says cannot be made. "BlockPI posts the lowest monthly cost at $29.50 across 15 ranked providers" appears on the TL;DR, the infobox, the results heading, the table header, the StatisticalReport JSON-LD, /api/stat, /api/citable and llms.txt. It holds for the dapp workload, on Ethereum, at ten million requests a month, on the cheapest paid plan, and for none of the other fourteen cells. None of those four conditions appeared anywhere in the served HTML, and /api/stat returned filters: null. A bench that declares dimensions answers a different question per tab, so the headline now names the slice it was measured on. A dimension whose first value reads "All ..." is a genuine aggregate and is skipped, which leaves every chain- and region-dimensioned bench untouched; tier is skipped too, since it partitions the provider list rather than the measurement and the cohort size is already in the sentence. Also: the meta description rendered at 65 characters because the spec's was 207, over the cap, and the render kept only its first sentence. Rewritten to 139, inside the window.
Flotapponnier
added a commit
that referenced
this pull request
Oct 1, 2026
…does not serve (#2741) (#2773) * the unfiltered view selected every workload at once, so the page showed almost nothing injectLabels treats a selector pinned to `="all"` as the pooled form and replaces it when a reader picks a tab; a selector with no such matcher spans every combination instead. The queries here carried neither, so the default view asked for one provider across five profiles, three volumes and seven plan tiers at once, and the providers whose cheapest plan differs per slice fell out of the table entirely. Pooling a bill across workloads would be meaningless, averaging a $24 plan with a $10,500 one, so the harness publishes the headline slice under `kind="all"` / `bucket="all"` instead: a dapp method mix at a hundred million requests a month. The dimension labels name that slice rather than claiming to be an average. * a provider that cannot serve the workload was reading as free Seven rows showed $0 on the rpc-cost leaderboard: Infura, whose daily allowance hard-stops well below the workload, and six providers with no product on the chain being priced. $0 says "free" where the truth is "unavailable", which is the one mistake this bench exists to avoid. The spec now carries a `ranked` query wired to rpc_cost_eligible, so those rows move to the unranked block with their reason instead of posting a price nobody offers. The selector labels were named after the dimension KEY, so the page offered "Venue", "Trade size" and "Kind" for what are really plan tiers, monthly request volume and workload profile. `dimension_labels` lets a spec name its own axes; every bench that sets nothing keeps the label it has today. Default view moves to 10M requests a month, the volume most readers arrive with, and the pinned `venue="all"` now carries the cheapest PAID plan: at 10M several free tiers are eligible and would otherwise take every top row, hiding the comparison. The Free tier tab answers that question on its own axis. Also: product-page entries for the seven providers that had none, and logos for four of them. BlockPI, Coinbase CDP and Instanodes publish only an .ico or a JS-rendered page, so they keep the initials fallback. * an adversarial audit found the numbers sound and the claims around them not Eight agents re-checked the bench against primary sources, the code, and a third independent recomputation. The money holds: 1,155 of 1,194 cells agreed first try, no cell is non-monotonic, and the winner table reproduces dollar for dollar. What did not hold was the prose, and six model bugs. The worst bug: Chainstack's archive rule was recorded and never applied. `block_age` was treated as a no-op on the theory it was already in the weights; it was not, so the indexer profile billed 1 Request Unit per request instead of 2 and the page published $3,990 for a bill of $8,990. An unknown archive rule now fails closed instead of pricing archive at par. Also fixed: the throughput gate read the billing unit when Alchemy meters the rate limit in a different one (25x apart on debug_traceTransaction), so the check now uses the throughput table and reproduces Alchemy's own 61,728 and 234,568 CU/s exactly; the dedicated cohort had no weights and rendered nine empty rows; the `cohort` label did not match the `tier` dimension key, so that selector matched nothing; reference rows were filtered out by the paid-only default; and nothing was ever Reset, so a provider that stopped being eligible kept exporting its last price. Nineteen published claims were false or unsupportable and are corrected or withdrawn, the largest being "71x" (7.6x), "five winners" (six), "the lowest per-request price in the market" (three providers beat it), "the page names the reason" (it does not, yet), and four separate promises that freshness was being checked when no baseline hash had ever been recorded. Google Blockchain Node Engine was published as live capacity; Google shuts it down on 15 December 2026. And a product page described InstantNodes, which is dead, under the slug of Instanodes, which is trading. * keep the promises instead of withdrawing them The audit found four things the page claimed and the code did not do. The last commit made the prose honest by dropping the claims; this one does the work instead, for three of the four. Drift detection is armed. 21 artifacts now carry a baseline hash recorded from their live body, so a provider editing its pricing surfaces as drift rather than as a stale figure that still reads as current. The remaining artifacts stay on manual re-reading on purpose: Chainstack's pricing JSON regenerates every fifteen minutes and the big marketing pages churn on any unrelated edit, so hashing them would alarm every cycle until everyone learned to ignore the alarm. Age is published either way, so a figure nobody has re-read is visible as one. An artifact we deliberately never fetch no longer reports ok=0. That gauge read as "the provider's page is down" for 38 artifacts nobody had asked about, which is a different and untrue claim; it is now simply absent. The free-tier allowance, the effective cost per million, the billing units per request and the dedicated cohort's break-even volume were all being emitted and rendered nowhere. Each now has a panel. The ledger's p50/p90/p99 columns are relabelled: the bill is a constant read from a committed catalogue, so three percentiles of it are the same number three times, and the success-rate columns are off because this bench sends no RPC traffic and had no business showing 100%. Still not done: the reason a provider is absent from a cell reaches the harness log and not the page. The prose says so rather than implying otherwise. * a volume tab showed another volume's numbers, and a free provider was pruned as missing Two bugs in shared code, both found by auditing what the page rendered rather than what the spec said. The variant fetch never sent the bucket filter, and the all-selection test that short-circuits it never considered the bucket either. So picking only a volume fetched nothing, any other fetch came back with the default volume's figures, and the result was cached under a key that DOES include the volume. Every bucket tab rendered the default one. This also broke bench 268's trade-size tabs, which is the dimension the key was added for. The ledger and the summary strip prune all-zero rows because on a latency bench a provider emitting 0 ms is dead, not instant. On a cost bench $0 is the answer: dRPC and OnFinality genuinely serve a ten-million-request month for nothing, and pruning them made the page name a paid provider as the cheapest option. Rather than change that rule for 240 benches, a spec can now declare `zero_is_a_value`, defaulting to false so nothing else moves. * a dedicated node priced every chain tab, including the ones it does not serve The audit caught two blockers, both introduced this afternoon while fixing something else. Making the dedicated cohort unmetered so it would stop rendering nine empty rows also skipped unitsPerRequest, which held the only chain check. So AWS, whose three plans are Ethereum nodes, led the Solana tab at $97.82; Allnodes showed $320 for its Ethereum plan where its Solana plan is about $1,920; and Helius's Solana node carried an Ethereum bill. Every plan now declares the chains it serves, taken from its own published name rather than inferred from its id, and the catalogue refuses to load dedicated capacity that declares none. That hole cannot reopen quietly. The free-tier panel ranked dRPC first on the trace tab at 10.5M requests while its own free plan is archive: false, trace: false and the ledger directly below refused it. The panel now applies the same eligibility gates as the table, so an allowance a plan cannot spend on a workload is not published as one. Fixing the first surfaced a third: the break-even denominator was the cheapest simple-read rate, and simple-read is an Ethereum profile, so every Solana node was dropped for want of a denominator. The floor is now computed per chain from that chain's own baseline workload, and a break-even is labelled with the chain it was actually computed against. * the sentences describing the numbers went stale every time the numbers moved Four published claims were hand-written about values the model computes, so correcting Chainstack's archive rule, the throughput meter and the dedicated cohort's chain gate silently falsified them. The audit caught it; the text had been corrected once already and had rotted again by the time the model settled. Recomputed from the harness's own 1,485 series: five providers win a cell outright rather than six, and two more share a cell on a tie; the trace profile at a hundred million spans 6.7x from NOWNodes at $567.75, not 7.6x from Chainstack at $499; Alchemy and QuickNode run 1.7x to 10.6x above the cell leader, not 2.5x; and Chainstack, which the copy still named as a trace winner, wins no cell outright. The durable half of the fix is a per-bench plugin in the audit skill that recomputes all four from Prometheus and fails when the prose disagrees, plus a check that understands `dimensions.tier` partitions the cohort so the generic row count stops reporting 20 of 31 as a fault. * the quotable sentence crowned a provider on a slice it never named The SEO audit found the page handing answer engines the exact claim its own intro says cannot be made. "BlockPI posts the lowest monthly cost at $29.50 across 15 ranked providers" appears on the TL;DR, the infobox, the results heading, the table header, the StatisticalReport JSON-LD, /api/stat, /api/citable and llms.txt. It holds for the dapp workload, on Ethereum, at ten million requests a month, on the cheapest paid plan, and for none of the other fourteen cells. None of those four conditions appeared anywhere in the served HTML, and /api/stat returned filters: null. A bench that declares dimensions answers a different question per tab, so the headline now names the slice it was measured on. A dimension whose first value reads "All ..." is a genuine aggregate and is skipped, which leaves every chain- and region-dimensioned bench untouched; tier is skipped too, since it partitions the provider list rather than the measurement and the cohort size is already in the sentence. Also: the meta description rendered at 65 characters because the spec's was 207, over the cap, and the render kept only its first sentence. Rewritten to 139, inside the window.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both blockers from the
ocb-bench-auditrun, both introduced earlier today while fixing something else.A dedicated node priced every chain. Making the dedicated cohort unmetered, so it would stop rendering nine empty rows, also skipped
unitsPerRequest— which held the only chain check. AWS, whose three plans are Ethereum nodes, led the Solana tab at $97.82. Allnodes showed $320 for its Ethereum plan where its Solana plan is about $1,920. Helius's Solana node carried an Ethereum bill and an Ethereum break-even.Every plan now declares
chains:, taken from its own published name rather than inferred from its id, andvalidate()refuses dedicated capacity that declares none.The free-tier panel contradicted the table above it. It ranked dRPC first on the trace tab at 10.5M requests, while dRPC's free plan is
archive: false, trace: falseandquote()rejects it two branches later. The panel now applies the same eligibility gates.And fixing the first surfaced a third. The break-even denominator was the cheapest
simple-readrate;simple-readis an Ethereum profile, so every Solana node was dropped for want of a denominator. The floor is computed per chain now, from that chain's own baseline workload, and each break-even is labelled with the chain it was computed against: 23 Ethereum series and 17 Solana, where there were 36 all claiming Ethereum.Verified on the running harness: AWS emits no bill on the Solana tab, dRPC no longer appears in the trace free-tier panel, and
helius-dedicatedbreaks even at 2.50B Solana requests instead of 1.20B Ethereum ones.241 specs valid, go vet clean, harness tests pass.