Skip to content

five RPC pages were measuring endpoints that had stopped answering - #2935

Merged
Flotapponnier merged 1 commit into
devfrom
fix/rpc-dead-endpoints
Oct 9, 2026
Merged

Flotapponnier merged 1 commit into
devfrom
fix/rpc-dead-endpoints

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

five RPC pages were measuring endpoints that had stopped answering

A sweep of the 64 thin and unread RPC pages probed all 159 configured
endpoints with the call each chain actually speaks, and 14 pages had at
least one dead endpoint. The dominant cause is not random rot: PublicNode
stopped serving five of our chains. The identical request still succeeds
on ethereum-rpc.publicnode.com and 404s on moonbeam from the same edge
with DNS resolving, so this is a withdrawal, not an outage.

Repaired here, each replacement probed with eth_getBlockByNumber or the
chain's own method, kept only after returning the same head as another
source on 10 of 10 tries:

moonbeam publicnode and tenderly 404, and the Foundation endpoint
refuses the TLS handshake from two independent stacks
(LibreSSL 3.3.6 and OpenSSL 3.5.5), so it cannot be
measured from our probes whatever the cause. Left drpc
alone, added onfinality and unitedbloc.
moonriver same PublicNode withdrawal, same Foundation refusal. Added
drpc and unitedbloc, both of which the spec had excluded in
August for a dead route and dead DNS and which answer again.
astar 1RPC answers about 15% of the time, the same ok_rate that
already got it dropped from Oasis Sapphire. Added the Astar
Foundation endpoint and drpc.
cosmos-hub Lava returns 410 Gone, consistent with its removal in #2926.
Replaced with ecostake.
gravity Ankr requires a key on this chain now and answers 403, so it
can never produce a sample. Replaced with thirdweb.

Two things found in passing and fixed:

Moonbeam's intro claimed four endpoints and carried a clause with no
subject. Moonriver's intro still listed its August cohort and named two
providers as excluded that work today. Both rewritten against what is
actually measured.

gravity-official was published with no endpoint field at all, on a page
whose own description promises URLs for every no-key endpoint. Added
rpc.gravity.xyz, the host the harness has been probing all along.

Gravity keeps its "publicnode" provider label even though that row is the
Foundation's own host. Renaming it would start a new series and blank the
column for a full day, and the page already displays it correctly.

Not recovered, recorded here so the next sweep does not re-probe them:
dymension, nibiru, conflux, merlin, swellchain, regen and nolus have no
third keyless endpoint that answers, so they stay under the three-provider
gate.

A sweep of the 64 thin and unread RPC pages probed all 159 configured
endpoints with the call each chain actually speaks, and 14 pages had at
least one dead endpoint. The dominant cause is not random rot: PublicNode
stopped serving five of our chains. The identical request still succeeds
on ethereum-rpc.publicnode.com and 404s on moonbeam from the same edge
with DNS resolving, so this is a withdrawal, not an outage.

Repaired here, each replacement probed with eth_getBlockByNumber or the
chain's own method, kept only after returning the same head as another
source on 10 of 10 tries:

  moonbeam    publicnode and tenderly 404, and the Foundation endpoint
              refuses the TLS handshake from two independent stacks
              (LibreSSL 3.3.6 and OpenSSL 3.5.5), so it cannot be
              measured from our probes whatever the cause. Left drpc
              alone, added onfinality and unitedbloc.
  moonriver   same PublicNode withdrawal, same Foundation refusal. Added
              drpc and unitedbloc, both of which the spec had excluded in
              August for a dead route and dead DNS and which answer again.
  astar       1RPC answers about 15% of the time, the same ok_rate that
              already got it dropped from Oasis Sapphire. Added the Astar
              Foundation endpoint and drpc.
  cosmos-hub  Lava returns 410 Gone, consistent with its removal in #2926.
              Replaced with ecostake.
  gravity     Ankr requires a key on this chain now and answers 403, so it
              can never produce a sample. Replaced with thirdweb.

Two things found in passing and fixed:

  Moonbeam's intro claimed four endpoints and carried a clause with no
  subject. Moonriver's intro still listed its August cohort and named two
  providers as excluded that work today. Both rewritten against what is
  actually measured.

  gravity-official was published with no endpoint field at all, on a page
  whose own description promises URLs for every no-key endpoint. Added
  rpc.gravity.xyz, the host the harness has been probing all along.

Gravity keeps its "publicnode" provider label even though that row is the
Foundation's own host. Renaming it would start a new series and blank the
column for a full day, and the page already displays it correctly.

Not recovered, recorded here so the next sweep does not re-probe them:
dymension, nibiru, conflux, merlin, swellchain, regen and nolus have no
third keyless endpoint that answers, so they stay under the three-provider
gate.
@Flotapponnier
Flotapponnier merged commit 994c7b8 into dev Oct 9, 2026
2 checks passed
@Flotapponnier
Flotapponnier deleted the fix/rpc-dead-endpoints branch October 9, 2026 18:39
bakabhai993 pushed a commit to bakabhai993/OpenChainBench that referenced this pull request Oct 18, 2026
main and dev had diverged both ways: 180 commits on dev, 87 on main, and no
fast-forward since October. A dev to main pull request would have opened
with 32 conflicts and no reviewer able to say which side was right without
reading all of them. This merges main into dev first, where the resolution
can be validated on staging, and leaves the release itself trivial.

Every file resolved in dev's favour was checked rather than assumed:

  the nine keyed RPC specs   main's corrections are already there (the
                             Alchemy figures match on all nine), dev adds
                             Validation Cloud, the p95 and the WebSocket arm
  the recording rules        dev holds all 11 of main's and 12 more
  rpc-capabilities config    dev carries the repair main never got, ChainBench#2935,
                             which replaced four endpoints that had stopped
                             answering; taking main would restore them
  the Hyperliquid family     10 commits on dev against 5, every file larger,
                             and 34 metric names queried against 32
  katana, xrplevm            dev has one more provider each
  the three registries       no entry exists only on main
  stakeangle.png             dev's is the optimised one from ChainBench#2938

Two were not a side to take:

  bench-published.json       the union. It records when a page first
                             appeared, so main's six dates for benches dev
                             has since removed are as true as dev's eleven
                             new ones. Earliest date wins where both know a
                             slug.
  provider-registry.ts       dev's entries, main's types. The triage had
                             compared entries and called them identical,
                             which they were; the module's exports were not.
                             main also landed ProviderClaim and an optional
                             claims field, and lib/provider-claims.ts, its
                             test and a product-page call site came through
                             the merge and failed to compile against dev's
                             registry. Caught by tsc, not by reading.

Gates on the merged tree: 253 specs valid, tsc clean, eslint 0 errors,
594 tests, promtool 23 rules, go build and vet on rpc-keyed-latency,
rpc-capabilities, hl-fills-reduce and hyperliquid-frontends, and a full
next build generating 293 static pages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant