Skip to content

OKX on bench 001, and BNB onto the reference clock #2794

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Flotapponnier merged 2 commits into from
Oct 3, 2026
Merged

OKX on bench 001, and BNB onto the reference clock #2794

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ad33653
OKX's DEX feed is open after all, on a channel their docs do not ment…
Flotapponnier Oct 3, 2026
353ca10
the BNB column was measuring BNB's block time, not the providers (#2792)
Flotapponnier Oct 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 43 additions & 7 deletions benchmarks/aggregator-head-lag.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@

slug: aggregator-head-lag
number: "001"
title: Fastest crypto price API, live head lag across Mobula, Codex, GeckoTerminal, Serialized
title: Fastest crypto price API, live head lag across 5 data providers
seo_title: "Fastest crypto price API 2026: live head lag by chain"
seo_description: "{{best_name}} leads cross-chain head lag at {{best_p50}} (p50, 24h); per-chain leaders differ. Mobula, Codex, GeckoTerminal, Serialized measured live."
subtitle: Wall-clock head lag in seconds from on-chain swap event to API emission, measured live for Mobula, Codex, GeckoTerminal and Serialized on Base, BNB Chain, Solana and Robinhood Chain.
seo_description: "{{best_name}} leads cross-chain head lag at {{best_p50}} (p50, 24h); per-chain leaders differ. Mobula, Codex, GeckoTerminal, Serialized, OKX measured live."
subtitle: Wall-clock head lag in seconds from on-chain swap event to API emission, measured live for Mobula, Codex, GeckoTerminal, Serialized and OKX on Base, BNB Chain, Solana and Robinhood Chain. OKX is measured on Base and Solana, the two chains where the headline does not read its timestamps.

per_chain_explainer:
- slug: base
h2: "Lowest Base price feed head lag"
body: |
{{best_name:chain:base}} currently leads Base head lag at {{best_p50:chain:base}} (p50, 24h) across the 4 measured providers. Base targets a 2 s sequencer cadence on the OP Stack, so the chain itself sets a roughly 2 s floor below which no provider can publish: any latency you see beyond that floor is pipeline work (indexer, fan-out, edge). Measured by holding the sequencer's flashblock preconfirmation stream and timestamping each provider's emission of the same swap from us-east, eu-west and Singapore, so the zero point is the moment the trade becomes public rather than the hour its block will carry.
{{best_name:chain:base}} currently leads Base head lag at {{best_p50:chain:base}} (p50, 24h) across the 5 measured providers. Base targets a 2 s sequencer cadence on the OP Stack, so the chain itself sets a roughly 2 s floor below which no provider can publish: any latency you see beyond that floor is pipeline work (indexer, fan-out, edge). Measured by holding the sequencer's flashblock preconfirmation stream and timestamping each provider's emission of the same swap from us-east, eu-west and Singapore, so the zero point is the moment the trade becomes public rather than the hour its block will carry.
- slug: bnb
h2: "Lowest BNB Chain price feed head lag"
body: |
Expand Down Expand Up @@ -63,9 +63,9 @@ abstract: |
same event.

methodology:
- "Aggregators measured: Mobula, Codex, GeckoTerminal, Serialized."
- "Aggregators measured: Mobula, Codex, GeckoTerminal, Serialized on every chain; OKX on Base and Solana only. OKX's DEX feed cannot be scoped to a pool, so it is subscribed per token and matched to the bench pool by transaction hash; its off-pool trades are discarded rather than scored. It is absent from BNB because that chain's headline reads the provider's own timestamp and OKX sends whole seconds, and absent from Robinhood Chain because OKX does not index it."
- "Chains: Base, BNB Chain, Solana, Robinhood Chain."
- "Reference on BNB Chain and Robinhood Chain: archive nodes per chain, validated against block hashes. These carry no preconfirmation layer a provider consumes, and their chain-supplied timestamps sit within roughly 60 ms of the moment a trade is observable, so they keep the on-chain timestamp as the zero point."
- "Reference on BNB Chain: a node subscription we hold ourselves, every emission matched to it by transaction hash, the ruler Base uses. Until 2026-10-03 the column used each provider's own timestamp, assuming BNB's chain-supplied timestamps sat within roughly 60 ms of a trade becoming observable. Measured, that gap is 569 ms, so the old column tracked block time more than any pipeline: the four feeds sat within 9% of each other, against the common clock they spread over a factor of four."
- "Solana (since 2026-09-16): the headline is the lag behind the first feed to report the trade, our own node subscription included in the race. Solana has no on-chain timestamp with sub-second precision, so the timestamps providers send compare conventions: Mobula's `date` (its ingestion time) read as a constant 0.10 s, Serialized's `at` (blockTime, whole seconds) as 0.75 s, while against a common clock the two feeds are 10 to 30 ms apart."
- "Solana zero point: no RPC WebSocket we hold, public or keyed (mainnet-beta, Helius, Alchemy), precedes the providers' geyser feeds, so an absolute zero point is not available and the first observation is the ruler. Our node stays in the race: it validates the hash on chain and bounds the field whenever a feed is slower than a plain RPC subscription."
- "Race: every feed's arrival for a transaction is compared to the earliest feed's (ties within 5 ms count as first for each feed involved; trades only one feed reported are not scored). Published on every chain as `head_lag_first_share_pct`, the First to report view: the share of the last 24 h of trades a feed reported before every other feed."
Expand Down Expand Up @@ -132,7 +132,7 @@ faq:
- q: "Why does the Solana leader show a lag near zero?"
a: "Because on Solana the ruler is the first feed to report the trade, not the chain. Solana's blockTime has one-second resolution and nothing on chain marks the millisecond a trade became observable, so measuring each provider against the timestamp it sends compared clock conventions: one provider's field was its own ingestion time (a constant 0.10 s), another's the blockTime (0.75 s on average). Against a common clock the two are 10 to 30 ms apart. The First to report view answers the question directly: the share of trades each feed delivered first. Our own node runs in the race and only wins on a small share of trades, which tells you these push feeds beat a plain RPC subscription; an absolute number needs a geyser-grade reference, which no RPC WebSocket we tested provides."
- q: "How is head lag measured on OpenChainBench?"
a: "On Base the harness holds the sequencer's flashblock preconfirmation stream in every region and timestamps each transaction the first time it appears there; on BNB Chain, Solana and Robinhood Chain it holds canonical-tip archive nodes and uses the on-chain timestamp. Every swap is timestamped with millisecond precision. The harness then watches each provider's feed for the same event and computes the delta. p50, p90 and p99 are aggregated over 24 hours via Prometheus `quantile_over_time`. Errors and dropouts are tracked separately as a presence ratio so a fast feed that disappears does not silently rank above a slower feed that stays up."
a: "On Base the harness holds the sequencer's flashblock preconfirmation stream in every region and timestamps each transaction the first time it appears there; on BNB Chain it holds a node subscription and matches every provider emission to it by transaction hash; on Solana and Robinhood Chain it uses canonical-tip archive nodes. Every swap is timestamped with millisecond precision. The harness then watches each provider's feed for the same event and computes the delta. p50, p90 and p99 are aggregated over 24 hours via Prometheus `quantile_over_time`. Errors and dropouts are tracked separately as a presence ratio so a fast feed that disappears does not silently rank above a slower feed that stays up."
- q: "Which data provider should I pick for a real-time trading interface?"
a: "Read both the p50 and the p99 columns. p50 tells the typical experience, p99 tells the worst minute a user will see when a chain spikes or a region's path degrades. A 500 ms p50 with a 30 s p99 means most of the time the UI feels snappy and once per minute it stutters. The leaderboard splits both, plus the success rate column flags providers whose feed dropped during the measurement window."

Expand Down Expand Up @@ -297,3 +297,39 @@ providers:
p50: avg by (aggregator) (quantile_over_time(0.50, head_lag_seconds{aggregator="geckoterminal", region="sgp"}[24h])) * 1000
series: avg_over_time(head_lag_seconds{aggregator="geckoterminal", region="sgp"}[1h]) * 1000 unless (changes(head_lag_seconds{aggregator="geckoterminal", region="sgp"}[15m]) == 0 and count_over_time(head_lag_seconds{aggregator="geckoterminal", region="sgp"}[15m]) > 5 and on(chain, region) sum by (chain, region) (changes(head_lag_seconds{region="sgp"}[15m])) > 5)


# Base and Solana only, and both reasons are structural rather than
# provisional. BNB publishes `head_lag_seconds` as receiveTime minus the
# provider's own timestamp, and every OKX timestamp is a whole second
# (1,800 sampled across three chains, all ts%1000 == 0), so measuring it
# there would publish our own arithmetic as OKX's latency. Robinhood Chain
# is not in OKX's supported set at all.
#
# OKX also cannot scope a subscription to a pool, so it pushes every pool on
# a token and most of what it sends is discarded. That does not change what
# is scored: only emissions matched to the reference by transaction hash are,
# and the reference is the bench pool. Measured on the Base pool before
# shipping: 213 of 213 pool swaps present in OKX's stream.
- slug: okx
name: OKX
tag: WebSocket trades stream
formula: "Median seconds between an on-chain swap on the bench pools and the same trade arriving on OKX's DEX WebSocket trades stream (subscribed per token, matched to the pool by transaction hash), sampled every 15s over 24h. Base and Solana only."
queries:
p50: avg by (aggregator) (quantile_over_time(0.50, head_lag_seconds{aggregator="okx"}[24h])) * 1000
p90: avg by (aggregator) (quantile_over_time(0.90, head_lag_seconds{aggregator="okx"}[24h])) * 1000
p99: avg by (aggregator) (quantile_over_time(0.99, head_lag_seconds{aggregator="okx"}[24h])) * 1000
mean: avg by (aggregator) (avg_over_time(head_lag_seconds{aggregator="okx"}[24h])) * 1000
success: clamp_max(avg by (aggregator) (count_over_time(head_lag_seconds{aggregator="okx"}[24h]) / 2880), 1)
sample_size: sum(count_over_time(head_lag_seconds{aggregator="okx"}[24h]))
series: avg_over_time(head_lag_seconds{aggregator="okx"}[1h]) * 1000 unless (changes(head_lag_seconds{aggregator="okx"}[15m]) == 0 and count_over_time(head_lag_seconds{aggregator="okx"}[15m]) > 5 and on(chain, region) sum by (chain, region) (changes(head_lag_seconds{}[15m])) > 5 and on(aggregator, region) avg_over_time(ws_connected{aggregator="okx", chain=""}[15m]) > 0.8)
live_activity: sum(changes(head_lag_seconds{aggregator="okx"}[15m]))
regions:
- region: us-east
p50: avg by (aggregator) (quantile_over_time(0.50, head_lag_seconds{aggregator="okx", region="us-east"}[24h])) * 1000
series: avg_over_time(head_lag_seconds{aggregator="okx", region="us-east"}[1h]) * 1000 unless (changes(head_lag_seconds{aggregator="okx", region="us-east"}[15m]) == 0 and count_over_time(head_lag_seconds{aggregator="okx", region="us-east"}[15m]) > 5 and on(chain, region) sum by (chain, region) (changes(head_lag_seconds{region="us-east"}[15m])) > 5 and on(aggregator, region) avg_over_time(ws_connected{aggregator="okx", region="us-east", chain=""}[15m]) > 0.8)
- region: eu-west
p50: avg by (aggregator) (quantile_over_time(0.50, head_lag_seconds{aggregator="okx", region="eu-west"}[24h])) * 1000
series: avg_over_time(head_lag_seconds{aggregator="okx", region="eu-west"}[1h]) * 1000 unless (changes(head_lag_seconds{aggregator="okx", region="eu-west"}[15m]) == 0 and count_over_time(head_lag_seconds{aggregator="okx", region="eu-west"}[15m]) > 5 and on(chain, region) sum by (chain, region) (changes(head_lag_seconds{region="eu-west"}[15m])) > 5 and on(aggregator, region) avg_over_time(ws_connected{aggregator="okx", region="eu-west", chain=""}[15m]) > 0.8)
- region: ap-southeast
p50: avg by (aggregator) (quantile_over_time(0.50, head_lag_seconds{aggregator="okx", region="sgp"}[24h])) * 1000
series: avg_over_time(head_lag_seconds{aggregator="okx", region="sgp"}[1h]) * 1000 unless (changes(head_lag_seconds{aggregator="okx", region="sgp"}[15m]) == 0 and count_over_time(head_lag_seconds{aggregator="okx", region="sgp"}[15m]) > 5 and on(chain, region) sum by (chain, region) (changes(head_lag_seconds{region="sgp"}[15m])) > 5 and on(aggregator, region) avg_over_time(ws_connected{aggregator="okx", region="sgp", chain=""}[15m]) > 0.8)
9 changes: 7 additions & 2 deletions harnesses/aggregator-head-lag/cmd/script/base_flashblock_ref_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,18 @@ func TestTxHashFromRaw(t *testing.T) {

// Only the chains in referenceChains publish a reference-based headline;
// the rest must keep the provider's on-chain timestamp untouched.
//
// BNB moved into the set on 2026-10-03 after its reference was measured
// against the gate in pending_match.go. Robinhood and Solana stay out for
// different reasons: Robinhood has no measurement yet, and Solana has no RPC
// WebSocket that precedes the providers' geyser feeds, so it races instead.
func TestReferenceChainsScope(t *testing.T) {
for _, chain := range []string{"bnb", "robinhood", "solana"} {
for _, chain := range []string{"robinhood", "solana"} {
if referenceChains[chain] {
t.Errorf("%s must not be re-based on the reference", chain)
}
}
for _, chain := range []string{"base"} {
for _, chain := range []string{"base", "bnb"} {
if !referenceChains[chain] {
t.Errorf("%s must be re-based on the reference", chain)
}
Expand Down
5 changes: 5 additions & 0 deletions harnesses/aggregator-head-lag/cmd/script/head_lag_monitor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -826,6 +826,11 @@ func runHeadLagMonitor(config *Config, stopChan <-chan struct{}) {
wg.Add(1)
go runSerializedHeadLagMonitor(config, stopChan, &wg)

// Start OKX monitor. Base and Solana only, and token-scoped upstream:
// see the header of okx_head_lag_monitor.go for both reasons.
wg.Add(1)
go runOKXHeadLagMonitor(config, stopChan, &wg)

// Wait for all to finish
wg.Wait()
fmt.Println("[HEAD-LAG] All monitors stopped")
Expand Down
Loading
Loading