Skip to content

Birdeye's trades socket was open the whole time, on a host their docs never name - #2797

Merged
Flotapponnier merged 1 commit into
devfrom
feat/birdeye-head-lag
Oct 4, 2026
Merged

Flotapponnier merged 1 commit into
devfrom
feat/birdeye-head-lag

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Adds Birdeye as a sixth provider on bench 001, Base and Solana.

I said twice this was impossible. It was not.

Both times the conclusion came from probes that were testing something other
than what I thought: Node's built-in WebSocket does not send custom headers.
Every "Birdeye refuses us" result came from a client that never sent the Origin
and User-Agent I believed it was sending.

Rewritten in Go and swept one factor at a time against the live endpoint:

configuration result
no headers 403 Just a moment...
Origin only 403 Just a moment...
Origin + User-Agent 101 Switching Protocols

The gate is Cloudflare's default rule rejecting a client that declares no
User-Agent. No TLS fingerprinting is involved, so the uTLS machinery in
utls_codex.go is not needed — and the question I had raised about whether
defeating a bot protection was acceptable here does not arise: declaring a
User-Agent is configuring an HTTP client, not forcing a lock.

Two other things that each produced a wrong conclusion first

Wrong host. The socket is multichain-socket.birdeye.so, not the
public-api.birdeye.so their documentation names. Probing the documented host
is what made this look closed; that one needs a $199/month Premium plan.

Wrong format assumption. Frames are zlib-compressed JSON. DevTools labels
them "Binary message", which reads like protobuf until you notice the 78 9c
magic.

The recipe, in full

h := http.Header{}
h.Set("Origin", "https://birdeye.so")
h.Set("User-Agent", "<browser UA>")
conn, _, _ := websocket.Dialer{}.Dial(
    "wss://multichain-socket.birdeye.so/solana/socket-optimize", h)
conn.WriteJSON(map[string]any{"type": "SUBSCRIBE_TXS",
    "data": map[string]any{"address": token, "filter": map[string]any{}}})
// frames: zlib -> JSON; type TXS_DATA carries data[].txHash

WELLCOME (their spelling) and ACK arrive first and are not evidence of
anything; only TXS_DATA is.

Why Base and Solana only

Identical to OKX. blockUnixTime is in whole seconds, so a chain whose
headline reads the provider's own timestamp — anything outside referenceChains
and raceChains — would publish that quantisation as Birdeye's latency.
Robinhood Chain is not covered upstream.

Token-scoped like OKX, so most of what arrives is off-bench and discarded: only
emissions matched to the reference by transaction hash are scored. birdeye
joins tokenScopedAggregators so its unmatched emissions are not counted as
misses.

Verification

With the monitor's own decoder against the live endpoint, not a copy of it:

decoded 1170 trades, 1170 with a hash

Both chains confirmed: Solana returns base58 signatures, Base returns 0x
hashes. That check ships as TestBirdeyeLiveFeed, opt-in so CI never depends on
a third party. The OKX integration shipped broken twice because a hand-written
probe with its own copy of the struct said the feed was fine, so this one tests
what actually runs.

Six unit tests pin the header gate, both frame encodings, the live frame shape,
the frame types that must be ignored, the chain restriction, and the endpoint.

What a reviewer should push back on

This is undocumented frontend infrastructure, same bet as the GeckoTerminal
integration already in this harness. If it moves, the flow watchdog purges the
gauge and the column reads "Feed down" rather than a wrong number.

Coverage is unmeasured. For OKX I checked against eth_getLogs that the
feed carries every pool swap (266 of 266). I have not done that for Birdeye, so
whether it publishes a complete stream or a subset is open. Read
head_lag_ref_matches_total{aggregator="birdeye"} once this runs in staging
before trusting the column.

Deploy note. The head-lag harness runs on three Railway services, not the
VPS, and auto-deploy does not fire on push: railway redeploy --service <name> --from-source --yes on each.

Checks

  • go build ./... and go test ./cmd/script/ clean
  • pnpm validate 235 specs valid
  • pnpm exec tsc --noEmit clean
  • bun test src/ 430 pass, 0 fail

🤖 Generated with Claude Code

… never name

I said twice that Birdeye could not be reverse engineered. Both times I was
wrong, and for a reason worth recording: my probes were testing something
other than what I thought.

Node's built-in WebSocket does not send custom headers. Every "Birdeye
refuses us" result came from a client that never sent the Origin and
User-Agent I believed I was sending.

Rewritten in Go and swept one factor at a time against the live endpoint:

  no headers            403 "Just a moment..."
  Origin only           403 "Just a moment..."
  Origin + User-Agent   101 Switching Protocols

So the gate is Cloudflare's default rule rejecting a client that declares
no User-Agent. No TLS fingerprinting is involved and the uTLS machinery in
utls_codex.go is not needed, which also retires the question of whether
defeating a bot protection was acceptable here: declaring a User-Agent is
configuring an HTTP client, not forcing a lock.

Two more things that each produced a wrong conclusion first:

The socket is multichain-socket.birdeye.so, not the public-api.birdeye.so
their documentation names. Probing the documented host is what made this
look closed; that one needs a $199/month Premium plan.

Frames are zlib-compressed JSON. DevTools labels them "Binary message",
which reads like protobuf until you notice the 78 9c magic.

Base and Solana only, for the same two structural reasons as OKX:
blockUnixTime is in whole seconds, so a chain whose headline reads the
provider's own timestamp would publish that quantisation as Birdeye's
latency; and Robinhood Chain is not covered upstream.

Token-scoped like OKX, so most of what arrives is off-bench and discarded.
Only emissions matched to the reference by transaction hash are scored, and
birdeye joins tokenScopedAggregators so its unmatched emissions are not
counted as misses.

Verified with the monitor's OWN decoder against the live endpoint rather
than a copy of it: 1,170 trades decoded, 1,170 carrying a hash. That check
ships as TestBirdeyeLiveFeed, opt-in so CI never depends on a third party.
The OKX integration shipped broken twice because a hand-written probe with
its own copy of the struct said the feed was fine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Flotapponnier
Flotapponnier merged commit 591b0d2 into dev Oct 4, 2026
2 checks passed
@Flotapponnier
Flotapponnier deleted the feat/birdeye-head-lag branch October 4, 2026 12:38
Flotapponnier added a commit that referenced this pull request Oct 4, 2026
… never name (#2797) (#2798)

I said twice that Birdeye could not be reverse engineered. Both times I was
wrong, and for a reason worth recording: my probes were testing something
other than what I thought.

Node's built-in WebSocket does not send custom headers. Every "Birdeye
refuses us" result came from a client that never sent the Origin and
User-Agent I believed I was sending.

Rewritten in Go and swept one factor at a time against the live endpoint:

  no headers            403 "Just a moment..."
  Origin only           403 "Just a moment..."
  Origin + User-Agent   101 Switching Protocols

So the gate is Cloudflare's default rule rejecting a client that declares
no User-Agent. No TLS fingerprinting is involved and the uTLS machinery in
utls_codex.go is not needed, which also retires the question of whether
defeating a bot protection was acceptable here: declaring a User-Agent is
configuring an HTTP client, not forcing a lock.

Two more things that each produced a wrong conclusion first:

The socket is multichain-socket.birdeye.so, not the public-api.birdeye.so
their documentation names. Probing the documented host is what made this
look closed; that one needs a $199/month Premium plan.

Frames are zlib-compressed JSON. DevTools labels them "Binary message",
which reads like protobuf until you notice the 78 9c magic.

Base and Solana only, for the same two structural reasons as OKX:
blockUnixTime is in whole seconds, so a chain whose headline reads the
provider's own timestamp would publish that quantisation as Birdeye's
latency; and Robinhood Chain is not covered upstream.

Token-scoped like OKX, so most of what arrives is off-bench and discarded.
Only emissions matched to the reference by transaction hash are scored, and
birdeye joins tokenScopedAggregators so its unmatched emissions are not
counted as misses.

Verified with the monitor's OWN decoder against the live endpoint rather
than a copy of it: 1,170 trades decoded, 1,170 carrying a hash. That check
ships as TestBirdeyeLiveFeed, opt-in so CI never depends on a third party.
The OKX integration shipped broken twice because a hand-written probe with
its own copy of the struct said the feed was fine.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant