Repository navigation
OKX and Birdeye on BNB, and Robinhood ruled out by measurement - #2805
Merged
Merged
Conversation
Both monitors shipped covering Base and Solana, and both headers gave the same reason for leaving BNB out: the chain published head_lag_seconds as receiveTime minus the provider's own timestamp, and both providers send whole seconds, so the column would have carried our arithmetic rather than their latency. That reason expired in #2794, which moved BNB onto the reference clock. I wrote the exclusion and then retired its premise an hour later without connecting the two, and the stale reason stayed in two file headers, two published formulas and a methodology bullet. The feed was never the obstacle. Measured 45 s per chain on the non-stable leg of the bench's USDT/WBNB PancakeSwap V3 pool, before changing either table: birdeye /bsc/socket-optimize 825 trades, 825 with a hash okx chainId 56 938 trades, 938 with a hash BNB is pool-scoped (eth_subscribe logs on the bench pool address), so a token-scoped provider's off-pool emissions find no match and are dropped, exactly as on Solana. This adds a chain where the comparison is sound and does not touch the open question about Base, whose flashblock reference is chain-wide. Robinhood stays out on both, and now for a measured reason instead of one read off a chain list: birdeye /robinhood/socket-optimize WELLCOME frame, 0 trades in 45 s okx chainId 4663 acked, 0 trades in 40 s Both on the pool address and on its non-stable leg, while Base and BNB on the same code delivered hundreds. An invented chain name 404s where /robinhood/ does not, so the route exists and the chain is simply not indexed. Same ack-proves-nothing shape as the six pool-scoping attempts documented in the OKX header, which is why an accepted subscription is not counted as coverage anywhere in this bench. Three comments on main asserted things the code contradicts, and they go with it. The Birdeye header still told the next reader that "no TLS fingerprinting is involved and the uTLS machinery is not needed here", forty lines above the uTLS dialer that exists because it is and it was; errBirdeyeForbidden still asserted the egress-IP diagnosis that #2802 retired. Both now point at the fingerprint. Two guard tests fired on this change, which is what they were for, but they asserted the conclusion (OKX must not cover BNB) rather than its premise (OKX may cover a chain only while that chain reads a clock we hold). Re-derived as the coupling, so taking bnb back out of referenceChains now fails and names OKX as the reason, which the unconditional pair could not do. Published copy also stopped overstating two things. The cardinality bullet still counted 4 aggregators on 4 chains; it is 22 aggregator-chain pairs now. And Birdeye's coverage is disclosed rather than implied by the presence ratio: its stream carried 106 of 442 and 13 of 44 bench-pool swaps across two Base windows, about a quarter, so its percentiles describe the trades it reports and not the pool. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Flotapponnier
added a commit
that referenced
this pull request
Oct 4, 2026
The subtitle renders on the bench page and in the share card, and it said OKX and Birdeye are measured on Base and Solana next to a chain list of four. The redeploy that follows this makes it false. The first paragraph of the OKX comment block kept the expired reason too, and called it structural rather than provisional, which is wrong in exactly the way that matters: it was provisional and it expired when BNB joined referenceChains. #2805 rewrote the paragraph below it and left this one, so the block argued with itself. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Flotapponnier
added a commit
that referenced
this pull request
Oct 4, 2026
Eight PRs of bench 001 work went straight to main this week on "mets live
now", and the fast-forward back to dev that AGENTS.md prescribes after a
hotfix was never done. The two branches have since diverged, so a
fast-forward is no longer possible: dev carries its own merges of the same
work under different SHAs plus commits main does not have.
This ports the one bench, by file, rather than reconciling twenty commits.
Every line it removes from dev is a line main deliberately replaced:
Timestamp string dev still types the OKX millis field as a
string, which is the bug #2795 fixed: the
WebSocket sends a JSON number and 136 of 136
frames were rejected
plain websocket.Dialer replaced by the pinned Chrome ClientHello in
#2802, without which Cloudflare 403s every
dial from Railway
the 403 retry loop #2799 gave it a circuit breaker after it
hammered a third party from three regions
TestOKXDoesNotCoverBNB re-derived as a coupling in #2805
The spec comes with it, which is the operational reason this cannot wait:
the materialize worker clones dev at BUILD time, so until this lands a
worker rebuild would pick up a spec that still says OKX and Birdeye are
measured on Base and Solana only.
Verified on dev's base, not main's: go build ./... and
go test ./cmd/script/ -run 'Birdeye|OKX|Reference|Chain' pass, pnpm validate
reports 235 specs valid.
The wider divergence is untouched and still needs a decision.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both token-scoped providers shipped on Base and Solana, excluding BNB because
that column published the provider's own whole-second timestamp. #2794 moved
BNB onto the reference clock an hour later, which retired the reason, and the
stale reason stayed in two file headers, two published formulas and a
methodology bullet.
Measured before changing either table
45 s per chain, on the non-stable leg of the bench's USDT/WBNB PancakeSwap V3
pool:
/bsc/socket-optimizechainId 56BNB's reference is
eth_subscribelogs on the bench pool address, so it ispool-scoped like Solana: a token-scoped provider's off-pool emissions find no
match and are dropped. This adds a chain where the comparison is sound, and
deliberately does not touch the open question about Base, whose flashblock
reference is chain-wide.
Robinhood: ruled out by measurement, not by a chain list
/robinhood/socket-optimizechainId 4663Both on the pool address and on its non-stable leg, while Base and BNB on the
same code delivered hundreds. An invented chain name 404s where
/robinhood/does not, so the route exists and the chain is simply not indexed. Robinhood
keeps four of the six feeds.
Same ack-proves-nothing shape as the six pool-scoping attempts already
documented in the OKX header, which is why an accepted subscription is never
counted as coverage in this bench.
Three comments on main that contradict the code
involved and the uTLS machinery is not needed here", forty lines above the
uTLS dialer that exists because it is and it was.
errBirdeyeForbiddenstill asserted the egress-IP diagnosis that the Go toolchain was the fingerprint Cloudflare refused #2802retired.
The two guard tests
They fired on this change, which is what they were for. But they asserted the
conclusion (OKX must not cover BNB) rather than its premise (OKX may cover a
chain only while that chain reads a clock we hold). Re-derived as the coupling:
take
bnbback out ofreferenceChainsand the test now fails naming OKX asthe reason, which the unconditional pair could not do.
Published copy
aggregator-chain pairs x 3 regions = 66.
ratio: 106 of 442 and 13 of 44 bench-pool swaps over two Base windows, about
a quarter, so its percentiles describe the trades it reports and not the pool.
Gates
go build ./...andgo test ./cmd/script/ -run 'Birdeye|OKX|Reference|Chain'pass;
pnpm validatereports 235 specs valid. Both formulas were over the 240character cap on the first attempt and were shortened, with the detail moved
into methodology bullets.
Not in this PR
on the bench pool alone. That affects OKX's already-published Base column and
is a separate decision.
devat build time, so it needs a rebuildbefore either provider's new chain appears on the page.
devis 11 commits behindmainon this bench: OKX on bench 001, and BNB onto the reference clock #2794 through the Go toolchain was the fingerprint Cloudflare refused #2802 all wentto
maindirectly and the fast-forward back todevwas never done.🤖 Generated with Claude Code