Repository navigation
the Go toolchain was the fingerprint Cloudflare refused - #2802
Merged
Merged
Conversation
Birdeye 403s from Railway and connects from a laptop, with the same code, the same headers, the same proxy, the same variable. I spent an afternoon eliminating all of those and never compared the one thing that differs. The Dockerfile builds FROM golang:1.24-alpine. The laptop runs 1.27.1. Different Go versions emit different ClientHellos, and Cloudflare scores the fingerprint. One machine, one network, one code path, changing only the toolchain: Go 1.24.4 stock TLS 0/5 connected, all 403 Go 1.27.1 stock TLS 5/5 connected Go 1.24.4 uTLS 5/5 connected That also retires the diagnosis I was most confident about. The OVH VPS "worked" because I sent it a binary cross-compiled from the laptop, so it carried 1.27's hello. Nothing about the egress IP was ever involved, and the proxy this dialed through is not needed. Bumping the Dockerfile to 1.27 would work today and break silently the next time Go changes its hello, with nobody connecting the two events. So the hello is pinned explicitly, the way utls_codex.go already does for Codex. A second trap came with it: uTLS alone still failed, with neither a connection nor a 403. The Chrome preset advertises h2 in ALPN, the server selects it, and the upgrade returns an HTTP/2 SETTINGS frame that gorilla reports as `malformed HTTP response "\x00\x00\x12\x04..."`. Config.NextProtos does not win: the extension has to be edited inside the spec. And the live test was building its own websocket.Dialer, so it passed on 1.27 and failed on 1.24 while the monitor succeeded. Third time in this integration that a check with its own copy of the thing under test said something false. It now dials exactly the way the monitor does: 1,130 trades decoded under Go 1.24.4, all carrying a hash. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Flotapponnier
force-pushed
the
fix/birdeye-tls-fingerprint
branch
from
October 4, 2026 16:58
7015b2d to
e1c954c
Compare
Flotapponnier
added a commit
that referenced
this pull request
Oct 4, 2026
Both monitors shipped covering Base and Solana, and both headers gave the same reason for leaving BNB out: the chain published head_lag_seconds as receiveTime minus the provider's own timestamp, and both providers send whole seconds, so the column would have carried our arithmetic rather than their latency. That reason expired in #2794, which moved BNB onto the reference clock. I wrote the exclusion and then retired its premise an hour later without connecting the two, and the stale reason stayed in two file headers, two published formulas and a methodology bullet. The feed was never the obstacle. Measured 45 s per chain on the non-stable leg of the bench's USDT/WBNB PancakeSwap V3 pool, before changing either table: birdeye /bsc/socket-optimize 825 trades, 825 with a hash okx chainId 56 938 trades, 938 with a hash BNB is pool-scoped (eth_subscribe logs on the bench pool address), so a token-scoped provider's off-pool emissions find no match and are dropped, exactly as on Solana. This adds a chain where the comparison is sound and does not touch the open question about Base, whose flashblock reference is chain-wide. Robinhood stays out on both, and now for a measured reason instead of one read off a chain list: birdeye /robinhood/socket-optimize WELLCOME frame, 0 trades in 45 s okx chainId 4663 acked, 0 trades in 40 s Both on the pool address and on its non-stable leg, while Base and BNB on the same code delivered hundreds. An invented chain name 404s where /robinhood/ does not, so the route exists and the chain is simply not indexed. Same ack-proves-nothing shape as the six pool-scoping attempts documented in the OKX header, which is why an accepted subscription is not counted as coverage anywhere in this bench. Three comments on main asserted things the code contradicts, and they go with it. The Birdeye header still told the next reader that "no TLS fingerprinting is involved and the uTLS machinery is not needed here", forty lines above the uTLS dialer that exists because it is and it was; errBirdeyeForbidden still asserted the egress-IP diagnosis that #2802 retired. Both now point at the fingerprint. Two guard tests fired on this change, which is what they were for, but they asserted the conclusion (OKX must not cover BNB) rather than its premise (OKX may cover a chain only while that chain reads a clock we hold). Re-derived as the coupling, so taking bnb back out of referenceChains now fails and names OKX as the reason, which the unconditional pair could not do. Published copy also stopped overstating two things. The cardinality bullet still counted 4 aggregators on 4 chains; it is 22 aggregator-chain pairs now. And Birdeye's coverage is disclosed rather than implied by the presence ratio: its stream carried 106 of 442 and 13 of 44 bench-pool swaps across two Base windows, about a quarter, so its percentiles describe the trades it reports and not the pool. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Flotapponnier
added a commit
that referenced
this pull request
Oct 4, 2026
Eight PRs of bench 001 work went straight to main this week on "mets live
now", and the fast-forward back to dev that AGENTS.md prescribes after a
hotfix was never done. The two branches have since diverged, so a
fast-forward is no longer possible: dev carries its own merges of the same
work under different SHAs plus commits main does not have.
This ports the one bench, by file, rather than reconciling twenty commits.
Every line it removes from dev is a line main deliberately replaced:
Timestamp string dev still types the OKX millis field as a
string, which is the bug #2795 fixed: the
WebSocket sends a JSON number and 136 of 136
frames were rejected
plain websocket.Dialer replaced by the pinned Chrome ClientHello in
#2802, without which Cloudflare 403s every
dial from Railway
the 403 retry loop #2799 gave it a circuit breaker after it
hammered a third party from three regions
TestOKXDoesNotCoverBNB re-derived as a coupling in #2805
The spec comes with it, which is the operational reason this cannot wait:
the materialize worker clones dev at BUILD time, so until this lands a
worker rebuild would pick up a spec that still says OKX and Birdeye are
measured on Base and Solana only.
Verified on dev's base, not main's: go build ./... and
go test ./cmd/script/ -run 'Birdeye|OKX|Reference|Chain' pass, pnpm validate
reports 235 specs valid.
The wider divergence is untouched and still needs a decision.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Birdeye 403s from Railway and connects from a laptop, with the same code, the same headers, the same proxy, the same variable. I spent an afternoon eliminating all of those and never compared the one thing that differs.
The bug
Dockerfile:FROM golang:1.24-alpine. The laptop: 1.27.1. Different Go versions emit different ClientHellos, and Cloudflare scores the fingerprint.One machine, one network, one code path, changing only the toolchain:
That also retires the diagnosis I was most confident about: the OVH VPS "worked" because I sent it a binary cross-compiled from the laptop, so it carried 1.27's hello. Nothing about the egress IP was ever involved, and the proxy this dialed through is not needed.
Why pin rather than bump
Bumping the Dockerfile to 1.27 works today and breaks silently the next time Go changes its hello, with nobody connecting the two events. The hello is pinned explicitly instead, the way
utls_codex.goalready does for Codex.A second trap came with it
uTLS alone still failed, with neither a connection nor a 403. The Chrome preset advertises
h2in ALPN, the server selects it, and the upgrade returns an HTTP/2 SETTINGS frame that gorilla reports as:Config.NextProtosdoes not win here. The ALPN extension has to be edited inside the spec.And the test was lying
TestBirdeyeLiveFeedbuilt its ownwebsocket.Dialer, so it passed on 1.27 and failed on 1.24 while the monitor itself succeeded. Third time in this integration that a check carrying its own copy of the thing under test said something false.It now dials exactly the way the monitor does. Under Go 1.24.4, the version this image builds:
Checks
go build ./...andgo test ./cmd/script/clean on both 1.24.4 and 1.27.1gofmtclean on both touched filesStill open, unchanged by this
Coverage is 24.5%, reproduced across two independent collections (106/442 and 13/44). Birdeye publishes about a quarter of the bench pool's swaps, so the column still needs that disclosed before it can be trusted.
🤖 Generated with Claude Code