Repository navigation
a rank badge crowned a provider the bench publishes outside its ranking - #3402
Merged
Merged
Conversation
Measured on staging after the board had already stopped doing it:
/api/badge/rpc-api-coverage/alchemy?chain=robinhood
"#1/6 (Robinhood Chain), 100.0%"
/api/badge/rpc-api-coverage/blockmachine?chain=robinhood
"#2/6 (Robinhood Chain), 95.9%"
/api/badge/rpc-api-coverage/quicknode?chain=robinhood
"#4/6 (Robinhood Chain), 91.1%"
Two things are wrong at once. Alchemy leads that chain only because the one
method it was not asked, trace_block, is the one every other provider there
refuses, which is exactly why the board publishes it outside the ranking with
"asked on only 86% of cells". And the percentages are the ALL-CHAINS figures
under a chain label: Robinhood's real numbers are 85.7 for BlockMachine and
85.7 for Quicknode, not 95.9 and 91.1.
The second is a documented approximation. rankOfChain says so in its own
comment: the per-chain leader stash is forced to rank 1 and everyone else is
ordered by their unfiltered p50, "soft approximation, full per-chain
leaderboards live on the bench page chain tabs". The exact path exists
(rankOfCell, fed by a spec's rank_matrix_query) and bench 311 declares none, so
it falls through. Declaring one would fix the figures and NOT the crown,
because the crown is in the value rather than in the approximation.
This commit fixes the first only, which is the part that states something
false rather than something imprecise.
A row the bench publishes outside its ranking is not ranked here either. Three
rank paths exist and all three would have published it, so the check sits
before them rather than inside each. The answer is placeholderSvg, which
already exists for "no defensible position this cycle", names the bench and
claims no rank. The guard can only withhold a rank, never invent one.
Badges are embedded in other people's README files, so this is the one surface
where a false position outlives the page it came from.
No test: the route is a Next handler whose rank paths need the whole bench
object, and a mock thorough enough to exercise this is a bigger change than
the guard. Verified against the live case on staging instead, which is the
evidence that matters: the same URL that read "#1/6" must stop claiming a rank
while the board still shows Alchemy outside the ranking.
Flotapponnier
added a commit
that referenced
this pull request
Oct 29, 2026
… we have (#3405) Verified on staging after #3402 landed, which is how this surfaced: alchemy ?chain=robinhood "measurement pending" alchemy unscoped "measurement pending" quicknode ?chain=robinhood "#4/6 (Robinhood Chain), 91.1%" (control) The guard works: Alchemy no longer claims a rank and a ranked provider still answers. The wording does not. Alchemy's figure exists, 100.0 percent over the 12 cells we could ask, and the board publishes it with its cause. Only the RANK is withheld. placeholderSvg hardcodes "Measurement pending" in the aria-label, the <title> and the visible line, so my guard replaced a false rank with a false status, which is the same fault facing the other way. placeholderSvg now takes an optional status line. The default is unchanged, so the three callers that mean it keep saying "Measurement pending": a transient scoped miss, an unknown cell, a bench with no cell data. The unranked path passes "Not ranked". NO FIGURE IN THE BADGE, deliberately, and this is the part I changed my mind on My first version read `Not ranked · ${fmtUnit(providerRow.ms.p50, b.unit)}`. That is wrong for the same reason rankOfChain was: `b` is the unfiltered bench, so under a ?chain= request it would print a cross-chain value behind a scoped label. It happens to be invisible on this bench, because Alchemy reads 100.0 both all-chains and on Robinhood, and it would be a live defect on any bench where those differ. "Not ranked" is true on every scope and asserts no number; the board carries the figure. So: a badge that cannot defend a position says so, and says nothing else.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measured on staging after the board had already stopped doing it:
Two things are wrong at once. Alchemy leads that chain only because the one
method it was not asked, trace_block, is the one every other provider there
refuses, which is exactly why the board publishes it outside the ranking with
"asked on only 86% of cells". And the percentages are the ALL-CHAINS figures
under a chain label: Robinhood's real numbers are 85.7 for BlockMachine and
85.7 for Quicknode, not 95.9 and 91.1.
The second is a documented approximation. rankOfChain says so in its own
comment: the per-chain leader stash is forced to rank 1 and everyone else is
ordered by their unfiltered p50, "soft approximation, full per-chain
leaderboards live on the bench page chain tabs". The exact path exists
(rankOfCell, fed by a spec's rank_matrix_query) and bench 311 declares none, so
it falls through. Declaring one would fix the figures and NOT the crown,
because the crown is in the value rather than in the approximation.
This commit fixes the first only, which is the part that states something
false rather than something imprecise.
A row the bench publishes outside its ranking is not ranked here either. Three
rank paths exist and all three would have published it, so the check sits
before them rather than inside each. The answer is placeholderSvg, which
already exists for "no defensible position this cycle", names the bench and
claims no rank. The guard can only withhold a rank, never invent one.
Badges are embedded in other people's README files, so this is the one surface
where a false position outlives the page it came from.
No test: the route is a Next handler whose rank paths need the whole bench
object, and a mock thorough enough to exercise this is a bigger change than
the guard. Verified against the live case on staging instead, which is the
evidence that matters: the same URL that read "#1/6" must stop claiming a rank
while the board still shows Alchemy outside the ranking.