Repository navigation
the WebSocket arm goes from 138 million messages a month to about six - #3018
Merged
Merged
Conversation
QuickNode wrote to say the monthly limit was reached. Measured before touching anything, over 24h: QuickNode took 4,598,911 pushes, 138M a month, against 583,140 HTTP probe calls. The arm is 237 times the HTTP probe by event count. The guard we had was looking the other way. `rpc_keyed_quota_used_ratio` reads 1.17% for QuickNode because it counts HTTP calls and knows nothing about sockets, which is why nothing warned before the vendor did. And this is not one vendor's problem. Chainstack 124M, Validation Cloud 124M, Dwellir 112M, GetBlock 99M a month. QuickNode is simply the first to have written. Two cuts, measured rather than guessed. The three fastest chains leave the arm. Robinhood alone is 40% of all pushes, Solana 19%, Arbitrum 14%, against Ethereum's 0.4%: a chain producing a block every 100 ms pushes 120 times what a 12 s chain does, to answer the same question. That is 73% of the volume for a third of the pages. Their three specs go with them; none was ever on main, so no redirect is owed. What remains listens ten minutes in every hour. The window is derived from the wall clock, and that detail is the whole correctness of it: if two providers computed different boundaries they would listen to different blocks, each would be its own leader, every lead would read zero and the board would report that every provider is fastest. A silent failure worse than an outage, so `wsWindowAt` is pinned by a test that asks several cells at skewed instants and requires them to open together. A close on the window deadline is not a disconnect. `rpc_ws_disconnect_total` is a published result about a provider, and our own scheduler must not write to it; the read goroutine racing the deadline is covered too. The skip list is compiled in rather than read from the environment, because it exists only because a bill was reached, and a lost variable must not be able to restore one in silence. RPC_KEYED_WS_SKIP_CHAINS overrides it, "none" empties it, and a duty window as long as its period restores the continuous behaviour without a code change. Expected effect per provider: 138M -> about 6M a month, a 95% cut, with six of the nine benches intact. Gates: gofmt clean on the file touched, go build, go vet, 4 new Go tests, 253 specs valid, 561 site tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
the WebSocket arm goes from 138 million messages a month to about six
QuickNode wrote to say the monthly limit was reached. Measured before
touching anything, over 24h: QuickNode took 4,598,911 pushes, 138M a month,
against 583,140 HTTP probe calls. The arm is 237 times the HTTP probe by
event count.
The guard we had was looking the other way.
rpc_keyed_quota_used_ratioreads 1.17% for QuickNode because it counts HTTP calls and knows nothing
about sockets, which is why nothing warned before the vendor did.
And this is not one vendor's problem. Chainstack 124M, Validation Cloud
124M, Dwellir 112M, GetBlock 99M a month. QuickNode is simply the first to
have written.
Two cuts, measured rather than guessed.
The three fastest chains leave the arm. Robinhood alone is 40% of all
pushes, Solana 19%, Arbitrum 14%, against Ethereum's 0.4%: a chain producing
a block every 100 ms pushes 120 times what a 12 s chain does, to answer the
same question. That is 73% of the volume for a third of the pages. Their
three specs go with them; none was ever on main, so no redirect is owed.
What remains listens ten minutes in every hour. The window is derived from
the wall clock, and that detail is the whole correctness of it: if two
providers computed different boundaries they would listen to different
blocks, each would be its own leader, every lead would read zero and the
board would report that every provider is fastest. A silent failure worse
than an outage, so
wsWindowAtis pinned by a test that asks several cellsat skewed instants and requires them to open together.
A close on the window deadline is not a disconnect.
rpc_ws_disconnect_totalis a published result about a provider, and our own scheduler must not write
to it; the read goroutine racing the deadline is covered too.
The skip list is compiled in rather than read from the environment, because
it exists only because a bill was reached, and a lost variable must not be
able to restore one in silence. RPC_KEYED_WS_SKIP_CHAINS overrides it,
"none" empties it, and a duty window as long as its period restores the
continuous behaviour without a code change.
Expected effect per provider: 138M -> about 6M a month, a 95% cut, with six
of the nine benches intact.
Gates: gofmt clean on the file touched, go build, go vet, 4 new Go tests,
253 specs valid, 561 site tests.