Repository navigation
rpc-cost audit round 2: reference cohort unranked, unverified prices never lead - #3008
Merged
Merged
Conversation
…prices never lead - A tier the spec declares ranked: false takes no rank, win, badge or JSON-LD position. PublicNode was '#1 of 1' with a shareable badge on its product page under a methodology that says it is never ranked. Its card now shows its $0 with 'not ranked' (zero_is_a_value honoured). - The curve carries each provider's unverified plan ids. The calculator's headline and chart take the cheapest VERIFIED plan; Zeeve, Dwellir Unlimited, Validation Cloud Scale and Tatum stay listed with an 'unverified' badge, and the card names a cheaper unverified price when there is one. - Copy: drift is a maintainer alarm, the page reads a committed curve dated in its footer (abstract, methodology, README and harness comments said the page showed drift and age). Chainstack's debug/trace rule stated right, a chain-scope bullet, 'cheapest metered' on the trace finding, the calculator's reasons scoped to the dapp mix. - The 282 gate comment restated: its three original conditions are met.
Flotapponnier
added a commit
that referenced
this pull request
Oct 16, 2026
* rpc-cost: the three findings the audit left standing (#2821) * rpc-cost: the three findings the audit left standing Bench 282 has been gated since 2026-09-30 with two blockers and a major that nothing had been done about. All three are the same shape: a value that was never stated read as a value of zero or one, producing a plausible wrong number that eighteen tests passed straight over. 1. block_age multiplied the whole weighted total. Chainstack's archive surcharge is triggered by how old a block is, not by which method asked, and several methods already carry the doubled figure in their own weight: debug_traceTransaction is listed at 2 RU and the empirical pass measured 2 RU even at tip-10. Multiplying again charged trace 3.6 RU against a real 2.0, an 80 % overstatement. The surcharge raises the floor rather than scaling the bill, so the archive weight of a method is now max(its listed weight, chain default x multiplier): a base read goes 1 -> 2, a debug call already at 2 stays 2. The indexer profile is deliberately unchanged at 2.0, because every method in it is 1 RU and the old multiplier reached the same answer there; a fix that moved that number would be over-reaching. 2. The dedicated cohort could not be billed for requests. needed = 0 for all 37 plans, so the overage and max_units branches were unreachable. AWS AMB read 97.82 dollars at every volume against a real 1B bill of 3,097.82, and led every Dedicated cell on it. The root cause is that included_units: null means two opposite things here. GetBlock's dedicated node serves unlimited requests for its monthly fee; AWS AMB includes none and bills every request on top. Both write null. The plan's own overage rate disambiguates, so that decides rather than the cohort: 4 of 37 meter (AMB in three regions, Shyft legacy-scale), 33 stay flat and their allowance is read as unlimited rather than zero. Zeeve's published ceilings now bind too. 3. An unstated capability was read as a granted one. archive: null and trace: null passed both gates, because one tested for the string "false" and the other for a non-nil pointer. 27 of the 106 usage plans are null on each field, and NOWNodes won trace cells on a capability it makes no claim about. Now not ranked, and the reason distinguishes the two cases: "does not serve" is the provider's statement, "not published" is a gap in this catalogue, and only the second is supportable for a null. 13 providers carry an explicit trace: true, so the cells keep a field: 20, 19 and 13 eligible plans at the three volumes. Three published winners move, two of them exactly as the audit predicted by hand before any of this was written: trace @ 10M chainstack 49 was drpc 60 trace @100m chainstack 499 was nownodes 567.75 trace @ 1B drpc 6000 was nownodes 5677.50 (unstated trace) indexer @ 1B onfinality 3624 Six tests, R5 to R7 from the audit's list. Verified by discrimination rather than by passing: with the fixes reverted, four of the five model tests fail and R6 fails, while TestFlatDedicatedPlanStaysFlat passes both ways, which is what it is for. The bench stays gated. The remaining audit items are the circular R1-R4 in the checks plugin, the units and free-allowance columns rendering on 4 of 90 views, and the free tier panel, which is the next piece of work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * pin the exact-fit edge and the unit conversion Florent asked for a triple check on the values, calling out that Chainstack bills in RU. Three things came out of it. The verification standard for this bench is two independent implementations agreeing on every cell, and I had not re-established it after changing the model. Rebuilt as a Node scorer reading the same catalogue with js-yaml, written from the catalogue's field semantics: 2,490 quotes, 0 disagreements on tier, eligibility, units per request and monthly bill, and all 15 cell winners identical. That run also taught something about what independence means here. My first attempt re-typed the workload profiles into the second implementation and produced 66 false disagreements, all from two transcription errors in the dapp and solana-bot mixes. Profiles are INPUT data; retyping them buys no independence and only adds copying mistakes, so the Go side exports them and the scorer reads them. The arithmetic is what has to be written twice. Then the published winners turned out to sit exactly on an allowance. Chainstack's archive workloads cost 2 RU per request after the block_age floor, its tiers include 20M / 80M / 200M / 400M RU, and the buckets are 10M / 100M / 1B requests. Round times round lands on round: trace and indexer at 10M need exactly 20,000,000 RU, which is exactly Growth, and at 100M exactly 200,000,000, which is exactly Business. Verified by hand, then reproduced by the scorer. So 49 dollars is correct and knife-edge. 10.1M requests is 52 dollars, 11M is 79, 15M is 199. Pinned, because a change to the weights, the floor or the buckets must not move that edge in silence. And a correction. I reported that the free tier could not be ranked for want of an allowance metric. Wrong: rpc_free_allowance_requests exists, is computed by FreeAllowanceRequests, and is live with 53 series. I had queried {__name__=~"rpc_cost.*"} and the metric is named rpc_free_*. It matters because converting units to requests REVERSES the raw ranking, and the raw figures are what I had shown: onfinality 400,000 RU/day -> 12,000,000 requests/mo drpc 210,000,000 CU/mo -> 10,500,000 ankr 200,000,000 credits -> 1,000,000 Ankr looks second best in the cohort by allowance and near the bottom by what it buys, because it charges 200 credits a request against OnFinality's 1 RU. All three hand-checked and now pinned. The catalogue's own comment says it: 200M credits means nothing until you know a credit buys a two-hundredth of a request. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit 949e0a2) * free plans tie at $0, so bench 282 was ranking them by their overage bill (#2826) The cost board banded plans by price and gave free ones their own tab on the same axis. That axis cannot separate them: every free plan costs the same nothing, so what the tab actually ordered was the bill each one sends after the allowance runs out. That showed the wrong half of the field. Chainstack Developer meters overage at $20 per million, so a 10M workload put it on the free tab at $140, dearer than its own paid Growth plan at $49. Alchemy and Infura are hard_stop, so they served nothing, were ruled ineligible, and left the tab altogether. A reader comparing free plans saw the one that bills and not the two that stop. Bench 283 ranks them on the only axis that separates them: how many requests of a real workload the allowance buys, once credits, Compute Units and Request Units are divided by what one request costs in them. Fourteen providers, and the spread is 160x on the Ethereum dapp mix, from OnFinality's 12.0M to GetBlock's 75,000. The order changes with the workload, so the figure is published per workload and never averaged: an allowance averaged across a trace mix and a simple-read mix describes no workload anyone runs. The metric already existed. rpc_free_allowance_requests has been published since the cost bench was built, with a comment saying the free tier belongs on its own axis. It had no `all` aliases, so a board pinning kind="all" would have found nothing: injectLabels only replaces a selector pinned to `="all"`. The headline slice is now published under those aliases too, as a slice and not as a pooled average, with three tests pinning that no other slice claims `all` and that both headline constants name profiles the harness produces. QuickNode is not on the new board. Its Free trial runs one month and does not renew, so it is priced on 282 as an entry plan instead. Gated until the harness is redeployed, since the aliases ship with this change and the page would otherwise render an empty table. (cherry picked from commit 8d2e54d) * the Solana tab was empty, because a workload already belongs to a chain (#2828) Reported straight after shipping: most of the selectors return nothing. They do. kind=all simple-read indexer trace solana-bot chain=all 11 11 9 5 EMPTY chain=solana EMPTY EMPTY EMPTY EMPTY 13 The board carried two axes, workload and chain, and the profiles only ever had one. simple-read, dapp, indexer and trace are Ethereum; solana-bot is Solana. Picking Solana with the default workload asked for an Ethereum mix on Solana, which nothing publishes, so the table came back empty. Eight of the ten cells could never have been filled. The chain axis is gone and the chain is named in each workload label. The chain alias goes with it: with `chain="all"` aliased onto Ethereum, a query that pinned kind only would have matched two chains at once and the scalar read would have returned null, which is the quieter version of the same bug. All five tabs now carry data: 11, 11, 9, 5 and 13 providers. A provider missing from a tab is serving a chain that tab does not ask about, and the methodology says so rather than leaving a gap to read as an absence. The test gains the invariant that would have caught this before deploy: every workload resolves to exactly one chain. (cherry picked from commit 7b76aee) * the free-tier board gave dRPC a Solana allowance three lines under the measurement refusing it (#2835) dRPC's free plan answers error 35, "chain is not available on free plan", on Solana. That is measured, dated 2026-09-29, and written in the plan's own note in this catalogue. The board published a 10.5M request Solana allowance for it anyway. The gate was not missing. FreeAllowanceRequests already skips a plan whose archive is false, whose trace is false, or whose declared chains exclude the workload's, and its comment cites dRPC leading the trace panel as the reason those branches exist. The chain branch is conditional on the plan declaring chains, and this one declared none. Nor does the test every metered cohort leans on catch it: that one lives in unitsPerRequest, and dRPC prices Solana on its paid plans, so the units resolve and the free plan rides through on weights it is not entitled to. A plan-level chains list is the only thing that can say "the provider serves this chain, this plan does not", so dRPC's free plan now carries chains: [ethereum]. Two tests. One holds the general rule against the shipped catalogue: a free plan that declares chains publishes an allowance for no other. One pins the dRPC row itself, so an edit that drops the line fails here instead of quietly restoring the figure. Left open deliberately: ten other free plans declare no chains at all, so the same hole is open wherever a provider prices both chains and restricts one on the free tier. Closing it by requiring the field would fail the catalogue on all ten and take the bench down; closing it by assuming a default would invent data in a file whose worth is that every line is measured and dated. Those ten need the probe dRPC got. (cherry picked from commit 7d237e6) * a free allowance has no single value, so the board publishes the span (#2837) The workload tabs were five, and a reader asking which free RPC plan is biggest had to click all of them and hold the numbers in their head. The reason the tabs exist is real: providers meter in units, a request costs a different number of units per method, and BlockPI's one free plan buys 3.1M simple reads and 368k traces. But asking the reader to discover that by clicking is the wrong half of the deal. Each row now carries the two ends of its own span next to the headline: the least and the most generous workload that plan serves on the selected chain. The spread is visible without a click, and the seven plans that meter flat report floor equal to ceiling, which says outright that their allowance cannot be surprised by a workload. The simple-read tab is gone: it drew the same eleven plans as the dapp tab in the same order bar one mid-table swap, because it is that mix's optimistic edge rather than a different thing to build, and the ceiling column now reports its number on every row at once. Four tabs, five priced mixes. Three things had to be fixed to make the columns true: The weighted mix was summed over a Go map, so the same plan priced differently from one call to the next and a span computed in a second pass could exclude the figure it brackets. Mix order is fixed now. The mean was not exact in the uniform case. Ankr charges a flat 200 units for every method; averaging share*200 over the seven-term dapp mix returned 199.99999999999994, so the allowance came back as 1,000,000.0000000002 on one mix and 1,000,000 on the others, and a flat-rate plan was published as workload-sensitive. A shifted mean makes the uniform case exact. A 24h range selector outlives its own corrections. The chain alias was retired this morning, and the dead chain="all" series went on answering for 21 more hours: every headline scalar saw two series where the loader wanted one and returned null on all 14 rows, and dRPC's Solana allowance, whose fix shipped in #2835, kept being published by the shadow. The scalars look back 3h now, matching the freshness the page promises and the harness's hourly cycle, and they aggregate, so the next label change costs nothing. The 24h window stays where it belongs, on the chart. (cherry picked from commit 1d7c512) * the Solana tab crowned the one row the catalogue refuses to rank (#2840) Solana Foundation led the Solana workload at 25.92M, more than double OnFinality, above eleven plans with published allowances behind them. Its own catalogue entry says the figure is DERIVED, not published: 10 requests/second integrated over 30 days, a throughput ceiling on an endpoint Solana's docs call "not intended for production applications". The entry carries ranked: false and two caveats saying exactly this, and the methodology printed under the table says keyless public endpoints with no published allowance are excluded. Nothing read any of it. `ranked` was wired into the cost metrics' cohort label and never into FreeAllowanceRequests, so the page contradicted both the catalogue and its own stated method, and the contradiction sat at the top of a tab. The cost benches can show a reference row beside the ranked ones because they have other columns; here the allowance IS the ranking, so a row that cannot be ranked has nothing to show. 13 plans, 11 per chain. Two figures shown next to provider names were also wrong, found by checking every tag against the catalogue: OnFinality was tagged "25M OnFinality credits a month". It is 400,000 response units a DAY, verified against onfinality.io today, which is 12M a month and the number the board was already computing. The tag invented both the value and the unit name, on the row leading three of four tabs. BlockPI was tagged "100 RU/s and a daily cap". Its rate is 400 RU/s and its free tier is neither daily nor capped: a 50M RU package issued on the 1st that expires after 32 days and does not accumulate, pro-rated here to 46.875M over 30 days. The methodology bullet repeated the same error by listing BlockPI among the providers whose daily figure is multiplied by 30. Verified in passing, and correct: Alchemy's 30M CU against its live CU table (eth_call 26, getBalance 20, blockNumber 10, receipt 20, estimateGas 20, sendRaw 40, getLogs 60) gives 23.9 CU on the dapp mix and 1,255,230 requests, the published figure to the unit, with simple reads and the indexer mix also exact. dRPC's flat 20 CU per method and Chainstack's 1 RU full / 2 RU archive both confirmed against their docs. (cherry picked from commit 2454768) * bench 282 priced three volumes, and every reader is at a fourth (#2927) Bench 282 prices 10M, 100M and 1B requests a month, because a Prometheus series needs a fixed label. Nobody runs 10M requests a month on purpose, so a reader at 40M or 3B had to pick the nearest column and guess which way the error ran, and on a metered bill with allowances and overage bands it does not run one way. The harness now emits a cost curve and the page interpolates between its points. The arithmetic is not reimplemented: the slider and the gauges both go through cheapest(), one at emit time and one at scrape time, so they cannot drift into disagreeing about what a provider charges. Interpolation is exact rather than smoothed, because inside one plan and one overage band the bill is affine in the request count and the emitter puts a point at every corner. Finding those corners took three tries, each drawing a straight line across something that is not straight: the handover between two plans happens where their lines cross, a corner of neither (Chainstack $83.83 at 10M against $49); the dedicated cohort steps $150 to $300 by adding a node while the plan id never changes (GetBlock $295 against $150); and thinning identical points collapsed a flat run to its first point, deleting the corner where the allowance runs out. The test is the artifact against the model at the three published volumes, every provider, every workload: 246 priced points within a tenth of a percent. 1,338 points, 158 KB raw and 13 KB gzipped. Three findings came out of asking the model at volumes nothing had asked before, all of them a $0 that was not a price. The curve first carried all 43 catalogue entries, including the twelve the board drops as not comparable, which would have put 1RPC, POKT, Lava and Tenderly on a slider sitting above a ledger that does not list them. Three plans cost $0 because they end: QuickNode's one-month trial, NOWNodes' one-month Start, Tatum's lifetime 100k credits. All three are banded as paid plans by what they become, with a tier_note saying in so many words that they do not recur, and the model read the $0 anyway. At 10M a trial is already past its cap, so nothing surfaced it; at 100k both one-month trials led the board. Plans now carry non_recurring. The guard is recurrence, not the zero: Validation Cloud's Scale has no subscription and its first 50M CU are free, so it still reads $0. Syndica, Triton and Coinbase CDP told readers nobody publishes a unit cost for eth_blockNumber. They carry `ethereum: null` under weights because the chain is not served, the key exists so the lookup succeeded, and the first method then failed for want of a weight. Right verdict, wrong branch, and a log line until the curve started printing each reason beside a provider's name. No published figure moves: all 435 readings the three gauge volumes cover, across every provider and all five workloads, are identical before and after. pnpm validate now fails when the artifact falls behind the catalogue it was built from, and names the command that fixes it. Bench 282 is still in DEV_ONLY_BENCH_SLUGS, so this is staging only. (cherry picked from commit c56ce58) * audit of benches 282 and 283 against the providers' own docs (#2930) A deep audit of benches 282 (`rpc-cost`) and 283 (`rpc-free-tier`) against each provider's own documentation. The arithmetic came out clean; the **inputs and the prose did not**. Seven published figures were wrong, one of them on production. ## The model is sound - **78 of 78** live `units_per_request` values reproduce exactly from an independent Python re-implementation reading the same catalogue (profiles parsed out of `profiles.go`, never retyped — retyping produced 66 false disagreements last time). This re-establishes the two-implementation standard after the model changes of 2026-10-09. - **BlockPI's $29.50**, the default view's leader, rebuilt from its live docs: all eleven Ethereum RU weights verbatim, `$49 / 500M RU / 60 days` verbatim, `$0.01 = 50,000 RU` verbatim. 27.5 RU/request re-derived by hand; `27.5 x 10M = 275M RU`, less 250M pro-rated, 25M overage at $0.20/1M = **$29.50**. - **Chainstack's trace = 2 RU** confirmed against the live doc's "Always 2 RUs" section: `debug_*` and `trace_*` are 2 RU *regardless of block age*, so there is no path to 3.6 or 4. - **Alchemy's 23.9 CU/request** matches the live CU table method by method; `30,000,000 / 23.9 = 1,255,230`, bench 283's figure. - **NOWNodes $102.19** and **Tatum $287** both reconstruct from their live pricing APIs. - **All 435 cost readings** across the three published volumes are byte-identical before and after everything in this PR. ## Seven wrong published figures **1. On production: nine pages quote an Alchemy limit Alchemy retired in August.** Every keyed RPC spec disclosed "a Free plan key (30M CU per month, **25 requests per second**)". Our own catalogue quarantines that exact figure: *"www.alchemy.com/pricing still renders the stale '500 CU/s (25 rps)' and must not be published"*, citing `alchemyplatform/docs` commit `3e0a4485`, 2026-08-24. Live today: `| Base Throughput (CUs/second) | 300 |`. Replaced with Alchemy's own unit; a converted request rate depends on the method mix, which is bench 282's entire subject. **These nine are on `main`.** **2-5. Bench 282 credited NOWNodes with trace wins it was removed from on 2026-10-05.** Recomputed every cell from the live gauges: | the spec said | live | |---|---| | trace@100M "6.7x, from NOWNodes at $567.75" | **7.6x, Chainstack $499** → GetBlock $3,779 | | "NOWNodes … winning the largest trace cell" | NOWNodes wins **no** trace cell | | "dRPC … 5.7% above NOWNodes" | dRPC wins **outright** | | FAQ "NOWNodes and Syndica two each" | **Chainstack** and Syndica two each | Check R1 went green on the last one: it compares the *count* of outright winners and the maximum, never the names. **6. "21 artifacts are re-hashed on a loop"** in four places. 21 carry a baseline hash, but `checkFreshness` skips excluded providers and the 21st is Lava, `cohort: excluded`. 20 are polled, matching the 20 live `rpc_pricing_artifact_ok` series. **7. AWS published a break-even volume at which it costs twice the alternative.** AWS AMB bills `$97.82 + $3.00/1M` against an Ethereum floor of `$2.833/1M`: its own rate is *above* the floor, so it is dearer at every volume and the gap widens. The published figure was **34,528,768 requests**, where AWS bills $201.41 against $97.82 metered. The tell was visible in the data: the three AWS regions meter at $3.00, $3.41 and $4.81 and all three published the *same* break-even, because the formula never read the rate. Now solved piecewise — flat region `fee/floor`, metered region `fee/(floor − rate)` — and where no crossing exists the harness publishes nothing and logs why. **40 series → 37.** The 33 flat plans are untouched; Helius dedicated still reads 2,495,594,714 against a live 2,495,590,000. ## Two free plans ranked on workloads their own pages refuse Bench 283's trace tab carried five rows. Two providers publish plainly that they do not sell what was being ranked: - **Ankr**: the feature table reads `Trace ❌ ❌ ✅` and `Debug ❌ ❌ ✅` across Public/Freemium/Premium. The catalogue had `freemium: trace: true`, so Freemium was published **second** at 1,000,000 requests/month. - **Instanodes**: the Free card lists, under "Not included", "Archive node access". The catalogue had `archive: null`, and null is a no-op in both gates, so it was ranked on the **indexer and trace** tabs at 600,000. The same table corrects Ankr in the other direction: `Full and Archive data ✅` in all three columns, so Public *does* serve archive — the catalogue said `archive: false` with the note "NOT PUBLISHED", wrong twice over. Measured against the corrected catalogue: | tab | before | after | |---|---|---| | trace | 5 rows | **3** — Chainstack 1.5M, BlockPI 367,936, GetBlock 14,423 | | indexer | 9 rows | **8** — Instanodes out; Ankr stays, it does have archive | | dapp (default) | 11 rows | **11, unchanged** | Also: bench 283 promises it reads "only the rows whose monthly price is zero **and whose allowance renews**". The second half was true only by convention — the three non-recurring offers are banded `entry`, so `TierFree` happened to exclude them — while `periodScale("lifetime")` returns 1, so a one-off grant banded `free` would have published as a full monthly allowance. `FreeAllowanceRequests` now skips `NonRecurring` explicitly. And its `seo_intro` claimed Validation Cloud "meters flat … floor is their ceiling". It prices `eth_call` at 40 CU and `eth_getLogs` at 80; live floor/ceiling is 1,219,512 / 1,250,000 (ratio 0.9756, against exactly 1.0000 for the four that really are flat). The spec's own provider tag says "weighted per method" four lines below. ## Findings left open, not fixed here - **9 of 20 watched artifacts are in drift**, two of them under the headline $29.50. Verified: BlockPI's are **cosmetic** — GitBook injected an `llms.txt` preamble and an 18-line "Agent Instructions" footer on every page, with no priced value changed. Chainstack's and Alchemy's are non-price too. The hashes need re-baselining, ideally with normalisation, or the alarm trains everyone to ignore it — which is the exact failure the methodology says it avoids for the manual sources. - **Two of four metric panels are structurally empty.** `rpc_cost_units_per_request{kind="all"}` → **0 series** (the harness emits only the five profile ids; `freeAllowance` beside it uses `aliasesFor`), and `injectLabels` appends `bucket=`/`venue=` to a metric that has neither, so it fills on 4 of 75 views. `rpc_cost_breakeven_requests` carries no `kind`/`bucket`/`venue` at all → 0 on the default view, and `prom.scalar` returns null on multi-series anyway. The methodology promises units-per-request "so the arithmetic is checkable". - **Tatum's two load-bearing inputs no longer exist at their citation.** `$9/1M credits` (which is $108 of the published $287) and `allowance_period: lifetime` are both cited to `tiers.json`; `0.000009`, `per credit`, `overage` and `lifetime` are all absent from the live file, whose FREE object was edited 2026-10-06T07:07:10Z. Neither is necessarily wrong; neither is checkable where we say to check it. - **The bench 283 podium rests on unwatched sources**, and OnFinality's 12M assumes a unit weight of 1 on Ethereum while its own docs list Ethereum among the chains with "higher Response Unit weightings for more intensive methods", unpublished. That makes 12M a ceiling, and the indexer tab is `eth_getLogs`-dominated. - **5 of bench 283's 11 ranked plans are `confidence: uncertain`, including the whole podium**, and no confidence signal is published for any of them. ## Checks - `go vet`, `go test ./...` clean, 4 new break-even tests (the old formula's 34,528,768 pinned as the regression case, plus a catalogue walk that fails if a fifth metering plan appears unreviewed) - `bun test` 25 pass; `pnpm validate` ✓ 246 specs; `pnpm typecheck` clean - `src/data/rpc-cost-curves.json` regenerates byte-identical, which is itself evidence the paid board does not move Bench 282 and 283 are both in `DEV_ONLY_BENCH_SLUGS`. The nine chain RPC specs in commit 1 are **not** — that correction is live-facing and needs a cherry-pick to `main`. (cherry picked from commit e3dd32b) * two blank panels, OnFinality's published weights, and the last four logos (#2940) OnFinality publishes the weights we said it did not, and it was leading on them The catalogue gave every OnFinality method a weight of 1, with a caveat saying the real multipliers are unpublished: /support/response-units states the default is 1 "across the majority of our networks" and names Ethereum and Solana as exceptions "with higher Response Unit weightings for more intensive methods", without giving them. The numbers are one click away. Those network names are links. The index at /support/networks promises "supported methods and response unit weighting", and /support/ethereum carries a table headed "The current supported method set and their Response Units weighting". Read live and verified here: eth_call 1 eth_getBalance 1 eth_blockNumber 1 eth_getBlockByNumber 1 eth_getTransactionReceipt 1 eth_getLogs 3 eth_estimateGas 3 eth_sendRawTransaction 12 eth_getBlockReceipts 13 trace_block 2 debug_traceTransaction 30 So the dapp mix is 1.75 RU per request and the indexer mix is 5.00, not 1. The caveat called those cells a LOWER bound. They were an upper one. What it moves, measured by running the harness against the corrected table: bench 283, the free-tier board dapp OnFinality 12,000,000 -> 6,857,143, and dRPC takes the lead indexer OnFinality 12,000,000 -> 2,400,000, a 5x overstatement simple-read unchanged at 12,000,000, which is the one tab the old figure was right on, and the one the "160x" finding rests on solana-bot OnFinality leaves the board entirely bench 282, the cost board OnFinality goes from four outright cell wins to none new tally: BlockPI 5, Chainstack 5, dRPC 2, Syndica 2 four winners, not five; the most any one takes is five, not four one tie cell at $49 (Chainstack, Helius), not two trace@100M now spans 13.7x, Chainstack $499 to OnFinality $6,849 Every sentence that quoted those facts is rewritten from the live gauges: the seo_intro's winner count, finding 1's spread, finding 4's claim about which figures are fully published, and the FAQ's per-provider tally. Bench 283's "meter flat, floor is their ceiling" list drops OnFinality in both places it appears — it now has the second-widest span on the board, 2.4M to 12M. Solana needed a judgement. All 60 methods in the published table weigh 2 and the page declares no default, so `default` is null rather than 2. And sendTransaction is absent from the table altogether, which reads as unpriceable rather than as 2 — the same rule Tatum gets on archive and NOWNodes gets on trace. That is what removes OnFinality from the Solana bot profile; a weight the provider has not published is not one we may invent. If the row appears, the cell returns on its own. Three free-tier confidence flags also come off `uncertain`, each on a primary source rather than a judgement: onfinality/developer the cap behaviour IS published — HTTP 403 and WS -32403 "Daily response limit exceeded", plus "Additional Response Units: Not provided" chainstack/developer 25 RPS on three primaries; the lone 30 is a Warning aside in a billing-procedure page, still live, now named in the note instead of leaving the row hedged instanodes/free both caps on three surfaces, and the hard stop is sourced: a 429 at the tier ceiling with no overage rate published anywhere Two stay uncertain and should: dRPC publishes nothing about what happens on exhausting its 210M CU, and Blockdaemon's own docs disagree with themselves about the free allowance, 3M in an April page and an archived pricing card against 1M in the October plan table. Both notes now say exactly what is missing rather than hedging the whole row. Also here, the four providers that still rendered as initials chips. BlockPI mattered: it leads the cost board and was the only leader without a mark. syndica.png and triton.png turned out to have been in public/logos all along and simply never registered, which is a reminder that dropping the file is half the job. All four take the provider's own glyph as SVG. (cherry picked from commit 0a678fd) * the break-even query could not parse, Best $0 held the strip, and a price is not a distribution (#2943) the break-even query I shipped yesterday cannot parse, and the strip still said Best $0 Two findings from re-auditing bench 282 after the data fixes landed. The first is mine, from the day before. **#2940's break-even metric does not parse.** I changed it to `min by (provider) (rpc_cost_breakeven_requests)` so the panel could collapse a provider's eleven plan series into one, and I tested that string against Prometheus, where it works. The page does not send that string. The loader splices the provider filter at the metric's FIRST brace and, when there is none, appends it to the end: min by (provider) (rpc_cost_breakeven_requests){provider="zeeve"} -> 1:48: parse error: unexpected "{" An empty selector inside the aggregation fixes it, verified the same way: min by (provider) (rpc_cost_breakeven_requests{provider="zeeve",}) -> 1 series, 52,594,423 I verified the mechanism instead of what the page asks, which is the exact failure this bench keeps finding in itself. So `pnpm validate` now refuses a panel metric that aggregates without carrying a selector, and I checked the guard by reverting the fix and watching it fail. **"Best $0" still heads the default view.** Blockdaemon publishes no paid price at all — its three paid plans are confidence: unpublished — so it has no headline result. The load path promotes it to "live" on the strength of its companion panels, and `computeFieldStats` kept it because bench 282 sets `zero_is_a_value`. That flag says a zero CAN be an answer. It does not say every zero is one. With the all-zero guard switched off, the strip printed Best "$0" above a ledger whose leader is BlockPI at $29.50, and the Spread printed "-" because tailMin had become 0. The ledger itself never showed the row: it drops at the 5% success floor. Two halves of one page disagreeing again, with the wrong half on top. The test is now whether the row was READ, not whether it came out zero. A measured zero keeps its success rate and stays, which is what the flag exists for — Validation Cloud's Scale has no subscription and its first 50M compute units are free. An unmeasured one has no success rate and goes, the same treatment the ranked surfaces already give it. Five tests pin both directions, including the measured zero. (cherry picked from commit 7bcf2c8) * Helius said its price was sales-gated, the free-tier panel was blank, and BlockPI was 6.25% low (#2952) Three majors from the bench audits of 282 and 283. Two are a page saying the wrong thing; one is a wrong number. ## Helius said its price was sales-gated. It publishes a $49 plan. The slider block and the harness log both read `not ranked (price not published (sales-gated))`. Helius publishes Developer at $49; the real reason it carries no Ethereum bill is that it has **no Ethereum weights at all**. The order was wrong. `quote()` tests `confidence: unpublished` before it reaches the chain test inside `unitsPerRequest`, and `cheapest()` then keeps the reason belonging to whichever ineligible plan had the largest allowance. A provider that simply does not serve the chain reported a price problem. Whether a provider serves the chain is a **provider** fact and outranks any plan-level question about price, so `cheapest()` now asks it once, up front, for the usage cohort. The dedicated cohort is exempt: it bills whole requests with no method weights and keeps its own chain test in `quote()`. ``` helius, syndica, triton, coinbase-cdp → "provider does not price ethereum" blockdaemon → "price not published (sales-gated)" ``` Blockdaemon's sentence is unchanged because for Blockdaemon it is true: it has Ethereum weights and all three paid plans are `confidence: unpublished`. Both are now accurate, which is the whole job of a panel that exists to tell a missing price apart from a cheap one. ## The free-tier panel was blank on 70 of 75 views Same cause as the two panels fixed in #2940 — and I fixed those two and left this one. `rpc_free_allowance_requests` carries provider, plan, kind and chain, so appending `bucket` or `venue` matches nothing. A free allowance does not change with how many requests you intend to buy or which paid band you would land in, which is the point of ranking it separately. ## Every BlockPI figure on bench 283 was 6.25% low The free plan was `allowance_period: package_32d`, so 50,000,000 RU was pro-rated to 46,875,000 for a 30-day month. **The 32 days is how long one package stays valid, not how often one arrives.** The catalogue already quoted the sentence that settles it: *"a gift package distributed to registered users on the 1st of every month ... expires after 32 days, does not accumulate"*. A calendar month delivers 50M; the overlap is forfeited, not carried. Strict pro-rating would not have given 46.875M either — twelve packages over 365 days is 49.3M per 30. | | before | after | |---|---|---| | dapp | 1,704,545 | **1,818,182** | | simple-read (ceiling) | 3,125,000 | **3,333,333** | | trace (floor) | 367,936 | **392,465** | | Solana | 721,154 | **769,231** | The spec's quoted figures move with them (`3.1M against 368k` → `3.3M against 392k`); the factor of eight still holds at 8.49. ## Effect, measured Eleven of the 435 published cost readings differ from the 2026-10-09 baseline, and **all eleven are OnFinality** — the published-weights correction from #2940 flowing through the dapp and indexer mixes and withdrawing it from solana-bot. The BlockPI period fix moves **zero** paid readings: it belongs to the free plan, and the paid packages keep their own `package_60d` / `package_90d` pro-rations. BlockPI still leads bench 282 at $29.50. 536 tests, 246 specs, `go vet` and the harness suite clean. (cherry picked from commit d6b85b4) * rpc-cost: Dwellir is priced, Instanodes leaves, Validation Cloud is one product (#2996) Dwellir bills one credit per response on every method, archive, debug and trace included, where Chainstack bills the same archive read and every trace call at 2 RU. Priced from dwellir.com/pricing.md (2026-10-16), it leads 7 of the 15 cells of bench 282 outright and ties Chainstack at $49 on two more: $1,999 for a billion trace requests against dRPC's $6,000 and Chainstack's $8,990. Its Unlimited Nodes and the Ethereum archive cluster join the dedicated cohort as dwellir-unlimited, marked uncertain because the premium endpoint add-on is unpriced. Its Free plan excludes eth_getLogs while the paid plans serve it, which the per-provider weights table cannot say. Plans gain unserved_methods: a workload calling one is unavailable on that plan, on the cost board and in the free allowance. On bench 283 that leaves Dwellir's free plan with no workload to rank, and the methodology says so instead of showing an empty row. Instanodes and the defunct InstantNodes are removed from the catalogue, both specs and the registry; the prose counts move with them. validation-cloud folds into validationcloud, so the latency and price pages are one product page, and the logo is the square mark Validation Cloud supplied, redrawn as a vector. The unused 256 px PNG goes. (cherry picked from commit 75a0e29) * products: the five bench-282 plan slugs fold into their vendor's page (#2997) dwellir-unlimited, getblock-dedicated, chainstack-dedicated, quicknode-flat-rate and helius-dedicated are plans priced in their own cohort, not products. Each built a /products page beside the vendor's. They now 308 to the vendor and their appearance lands on its page. (cherry picked from commit df3fa75) * rpc-cost opens on one slider and an archive checkbox (#3000) Four rows of selectors (cohort, plan band, volume, workload) over a three-volume ledger made the reader assemble the answer. The page now opens on the calculator: slide to a monthly volume, tick archive if the calls read old state, and it names the cheapest plan that serves it, metered or flat rate, then ranks the rest. The archive curve is a new curve-only profile (the dapp mix with every call billed as archive): no gauge, no tab. The curve file now carries the catalogue's plan names, and the catalogue's as_of moves to the Dwellir edit. (cherry picked from commit b3ac9de) * rpc-cost: the calculator gets a chart and clickable rows; AWS and Google leave (#3002) AWS Managed Blockchain and Google's Blockchain Node Engine rent a node by the hour inside a general cloud rather than sell an RPC service, and both are closing. They move to the excluded cohort with the reason written, leave the spec and the registry, and a finding says why. The calculator: a recent-state / archive switch, the cheapest plan in an accent card that links to its product page and says what it saves over the next plan, a log-log SVG chart of the bill against volume (the cheapest plan at every volume plus the four runners-up, a shaded band where a free plan covers the month, click or drag to move the volume), and ranked rows that link to each product page with a log bar and the multiple of the cheapest. SVG coordinates and CSS widths are rounded so server and client print the same attribute. (cherry picked from commit 7443478) * rpc-cost audit fixes: per-workload grid, honest copy, panel carry guard (#3006) From the 2026-10-16 bench audit (verdict FIX THEN SHIP; every cell figure and all 15 winners recomputed and matched). - The page lost every Solana, indexer and trace figure when the selectors went. A fixed grid under the calculator gives the cheapest metered plan for each of the five workloads at 10M, 100M and 1B: the fifteen cells the copy counts (Dwellir 7, BlockPI 3, Syndica 2, three ties at $49), each linking to its product page. - Calculator copy said what the code does not: no findings are rendered, free plans are not in the list (paid only, bench 283 has them), and flat rate nodes can undercut the metered leader. Rewritten. - Spec prose described a ledger, a profile picker and an age display that do not exist; 20 providers and 20 polled artifacts were 18 and 23; Blockdaemon (sales-gated) and Coinbase CDP (Base only) leave the provider list since neither can fill a cell. - Worker: panel values were carried with neither the declared-slug guard nor the age ceiling the result carry has, so Instanodes' price and two Ethereum-only nodes' Solana break-even outlived their removal. Tests. - Product pages said 'data warming up' for Syndica, a two-cell winner, because the default view is Ethereum. They now say there is no Ethereum plan. - Harness: a body that fails mid-read no longer reports healthy, and a block_age rule of 0 is refused at load. (cherry picked from commit 1515c96) * rpc-cost: the nine drifting pricing artifacts re-read (#3007) Eight of nine changed in form only (new rows or sections, a markdown table turned HTML); every figure the catalogue uses was checked against the live page and matches. Hashes and verified_at updated so the drift alarm reads clean again. Tatum's tiers.json dropped its overage rate on 2026-10-12: the $9 per 1M credits is no longer published anywhere we can read. It stays as the last published figure and the 13 paid plans are marked uncertain. NOWNodes' caveat on Enterprise overage corrected (0.5 EUR per 100k, which the figure already used). No published number moves. (cherry picked from commit 03af352) * rpc-cost audit round 2: no rank for the reference cohort, unverified prices never lead (#3008) - A tier the spec declares ranked: false takes no rank, win, badge or JSON-LD position. PublicNode was '#1 of 1' with a shareable badge on its product page under a methodology that says it is never ranked. Its card now shows its $0 with 'not ranked' (zero_is_a_value honoured). - The curve carries each provider's unverified plan ids. The calculator's headline and chart take the cheapest VERIFIED plan; Zeeve, Dwellir Unlimited, Validation Cloud Scale and Tatum stay listed with an 'unverified' badge, and the card names a cheaper unverified price when there is one. - Copy: drift is a maintainer alarm, the page reads a committed curve dated in its footer (abstract, methodology, README and harness comments said the page showed drift and age). Chainstack's debug/trace rule stated right, a chain-scope bullet, 'cheapest metered' on the trace finding, the calculator's reasons scoped to the dapp mix. - The 282 gate comment restated: its three original conditions are met. (cherry picked from commit 00ff87e) * rpc-cost: calculator without the chart, three missing logos, audit round 3 (#3010) Design: the chart is gone. The calculator is one card: controls on a tinted band (volume in large type, the 10M/100M/1B marks as pills, the archive switch), then the cheapest verified plan with its next plan, the spread across plans and a product link, the next three as cards, and the rest as a compact list. Every row has a mark: a logo, or the provider's initial when there is none. Logos: Allnodes (its app icon), Shyft and Zeeve (marks cropped from their own wordmark SVGs; Zeeve on white so its dark dot survives the dark theme). Audit round 3: - sample_size counted 7 days of a series the harness withdrew, so a provider removed from a cell read 'unresponsive, 0% success' for a week (OnFinality on the Solana tab). It now uses the p50's 20 min window. - The dedicated cohort is ranked: false like the reference one: it ranked unverified flat-rate prices and gave Dwellir a '#1 finish' on $119. - The break-even panel pooled Ethereum and Solana into one minimum; it is API-only and wrong on every Solana tab, so it is removed. - The metric names its slice (dapp mix at 10M), so a product page rank is not read as the bench's verdict. - min(last_over_time(...)) is a latest value, not a 24 h median, and USD under $1,000 prints cents as two digits ($29.50, not $29.5). - Copy: per-call unit weights stated as such, plan bands say base fee, free-allowance absence explains Dwellir. (cherry picked from commit da1ca1e) * release: bench 282 rpc-cost goes to production Ungated after three audit rounds on 2026-10-16 (every published bill and all 15 cell winners recomputed and matched). Its gate conditions of 2026-09-30 are met: the mislabelled selector is gone, the GetBlock comments were corrected, Chainstack trace is 2.0 RU. Bench 283 rpc-free-tier stays off main: its spec, carried by the cherry-picked split commit, is removed here. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second audit round of bench 282 (2026-10-16). All 15 grid cells and two hand-recomputed cells ($29.50, $8,990) match; three majors fixed here.
ranked: falseon adimensions.tiervalue.providers.tsskips ranking such cohorts, so PublicNode loses its "drop journal framing #1 of 1" rank, win, badge card and JSON-LD position; its product card shows the $0 figure as "not ranked" (honouringzero_is_a_value).unverifiedplan ids per provider (catalogue confidence != verified). Calculator leader card and chart envelope use the cheapest verified plan; unverified plans stay in the list with a badge, and the card names a cheaper unverified price when one exists.zero_is_a_valuecomment, gate comment inremoved-benches.ts.Checks: pnpm validate, bun test src/ worker/ (568 pass), go vet + test, tsc, eslint.