Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 2 additions & 25 deletions benchmarks/evm-quote-latency.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ per_chain_explainer:
- slug: ethereum
h2: "Fastest Ethereum swap quote API"
body: |
{{best_name:chain:ethereum}} currently leads the Ethereum swap-quote field at {{best_p50:chain:ethereum}} (p50, 24h) across the 7 measured providers. Ethereum mainnet holds the deepest EVM liquidity (Uniswap v3 tick-range pools, Curve stable pools, Maker PSM), so route-search complexity is the dominant component of quote latency: aggregators trading off depth-of-search against time-to-first-byte show their architecture here more than on any other chain. Probes run every 60 seconds on a 5-pair basket from us-east, eu-west and Singapore.
{{best_name:chain:ethereum}} currently leads the Ethereum swap-quote field at {{best_p50:chain:ethereum}} (p50, 24h) across the {{count:chain:ethereum}} measured providers. Ethereum mainnet holds the deepest EVM liquidity (Uniswap v3 tick-range pools, Curve stable pools, Maker PSM), so route-search complexity is the dominant component of quote latency: aggregators trading off depth-of-search against time-to-first-byte show their architecture here more than on any other chain. Probes run every 60 seconds on a 5-pair basket from us-east, eu-west and Singapore.
- slug: base
h2: "Fastest Base swap quote API"
body: |
Expand Down Expand Up @@ -69,7 +69,7 @@ methodology:
- "Three regions (us east, eu west, sgp). Each is a separate Railway service writing the same metric family with a region label, so the page can filter to a single edge or aggregate across all three."
- "Latency is wall clock from request dispatch to last byte received, observed only on the happy path (HTTP 2xx and a parseable output amount). Failures land on dedicated counters (auth, throttle, no route, other) and pull the success gauge to 0."
- "Native asset sentinel is the canonical 0xEeeeeEeee form. Uniswap uses 0x0000... and Bebop substitutes the wrapped equivalent on the sell side. The adapter normalises per provider."
- "Provider auth: four providers need a key (Mobula in an Authorization header, Route as a Bearer token, 0x and Uniswap in a vendor header), and KyberSwap, Bebop, LI.FI, CoW, ParaSwap, Nordstern and OpenOcean answer a bare request. Enso takes an optional key to widen its rate bucket and still reads unavailable here because every probe comes back throttled. The key is not what buys the low end of this table."
- "Provider auth: three providers need a key (Mobula in an Authorization header, Route as a Bearer token, Uniswap in a vendor header), and KyberSwap, Bebop, LI.FI, CoW, ParaSwap, Nordstern and OpenOcean answer a bare request. Enso takes an optional key to widen its rate bucket and still reads unavailable here because every probe comes back throttled. The key is not what buys the low end of this table."

findings:
- "{{best_name}} currently leads at {{best_p50}} across all chains and regions, across {{count}} measured providers. Rankings shift per chain tab: a provider's edge placement and per-chain engine split matter more than its global average."
Expand Down Expand Up @@ -276,29 +276,6 @@ providers:
p50: avg(ocb:evm_swap_quote_latency_ms:p50_24h{provider="enso", region="sgp"})
series: histogram_quantile(0.50, sum(rate(evm_swap_quote_latency_ms_bucket{provider="enso", region="sgp"}[1h])) by (le))

- slug: 0x
name: 0x
tag: Swap API key, declares its own fee
formula: "Avg of per-(chain, region) 24h medians of evm_swap_quote_latency_ms{provider=0x} over the last 24 hours."
queries:
p50: avg(ocb:evm_swap_quote_latency_ms:p50_24h{provider="0x"})
p90: avg(ocb:evm_swap_quote_latency_ms:p90_24h{provider="0x"})
p99: avg(ocb:evm_swap_quote_latency_ms:p99_24h{provider="0x"})
mean: sum(ocb:evm_swap_quote_latency_ms:sum_rate_24h{provider="0x"}) / sum(ocb:evm_swap_quote_latency_ms:count_rate_24h{provider="0x"})
success: avg(ocb:evm_swap_quote_success:avg_24h{provider="0x"})
sample_size: sum(ocb:evm_swap_quote_latency_ms:count_increase_24h{provider="0x"})
series: histogram_quantile(0.50, sum(rate(evm_swap_quote_latency_ms_bucket{provider="0x"}[1h])) by (le))
regions:
- region: us-east
p50: avg(ocb:evm_swap_quote_latency_ms:p50_24h{provider="0x", region="us-east"})
series: histogram_quantile(0.50, sum(rate(evm_swap_quote_latency_ms_bucket{provider="0x", region="us-east"}[1h])) by (le))
- region: eu-west
p50: avg(ocb:evm_swap_quote_latency_ms:p50_24h{provider="0x", region="eu-west"})
series: histogram_quantile(0.50, sum(rate(evm_swap_quote_latency_ms_bucket{provider="0x", region="eu-west"}[1h])) by (le))
- region: ap-southeast
p50: avg(ocb:evm_swap_quote_latency_ms:p50_24h{provider="0x", region="sgp"})
series: histogram_quantile(0.50, sum(rate(evm_swap_quote_latency_ms_bucket{provider="0x", region="sgp"}[1h])) by (le))

- slug: paraswap
name: ParaSwap
tag: Trades as Velora, no key
Expand Down
14 changes: 1 addition & 13 deletions benchmarks/evm-quote-output.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ methodology:
- "Declared fees get their own column and are measured, not quoted: the figure is read out of the provider's own quote response on every tick. Only some declare one, and the column is blank for the rest. Compare it against the shortfall beside it rather than assuming they match, because they need not: a declared fee is what the provider takes, the shortfall is what the route and the fee together cost. A provider that declares nothing publishes no fee, which is not the same as charging nothing."
- "Not every row is an aggregator. CoW and Bebop are RFQ or batch-auction venues and LI.FI is a cross-chain relayer, so all three are listed with their numbers and kept out of the ranking. Route quotes Robinhood Chain alone and is kept out of the all-chains order for the same reason; its own tab carries the comparison."
- "An RFQ quote is not a router quote. A market maker prices its own spread and its settlement into the number it returns, and a small clip pays for both in full: Bebop gives up around a hundred basis points on a $1,200 swap, and the figure barely moves between chains whose gas differs by orders of magnitude, so it is the spread and not the gas. Both RFQ rows are listed and neither is ranked."
- "Gas is not subtracted yet. KyberSwap reports it in USD, 0x in wei, Nordstern in gas units, and the USD figures of three providers on the same pair at the same block differ by a factor of 3.5. Until gas is computed on our side rather than theirs, this page measures gross output."
- "Gas is not subtracted yet. KyberSwap reports it in USD, ParaSwap in wei, Nordstern in gas units, and the USD figures of three providers on the same pair at the same block differ by a factor of 3.5. Until gas is computed on our side rather than theirs, this page measures gross output."
- "Absent and why: 1inch requires a business account we do not have, OKX requires credentials not yet issued, Odos has published no volume for 30 days and its API returns 530, OpenOcean serves our probes a Cloudflare challenge, and Enso's free-plan credits are exhausted. None of them is excluded on performance."
- "Six pairs across five chains, one per 60 second tick, from us-east, eu-west and Singapore. Output does not vary meaningfully by region, measured at a 4.24 bps median spread against a market that drifts 30 to 37, so the regional split is a check on the probe rather than a dimension of the result."

Expand Down Expand Up @@ -168,18 +168,6 @@ providers:
sample_size: sum(ocb:evm_swap_quote_shortfall_bps:count_increase_24h{provider="paraswap"})
series: histogram_quantile(0.50, sum by (le) (rate(evm_swap_quote_shortfall_bps_bucket{provider="paraswap"}[1h])))

- slug: 0x
name: 0x
tag: Swap API key, declares its own fee
formula: "Median basis points of output given up against the best quote of the same tick, over 24h, averaged across the chains this provider covers."
queries:
p50: avg(ocb:evm_swap_quote_shortfall_bps:p50_24h{provider="0x"})
p90: avg(ocb:evm_swap_quote_shortfall_bps:p90_24h{provider="0x"})
p99: avg(ocb:evm_swap_quote_shortfall_bps:p99_24h{provider="0x"})
mean: avg(ocb:evm_swap_quote_shortfall_bps:mean_24h{provider="0x"})
sample_size: sum(ocb:evm_swap_quote_shortfall_bps:count_increase_24h{provider="0x"})
series: histogram_quantile(0.50, sum by (le) (rate(evm_swap_quote_shortfall_bps_bucket{provider="0x"}[1h])))

- slug: mobula
name: Mobula
tag: API key, routes multi-hop across named pools
Expand Down
7 changes: 6 additions & 1 deletion src/lib/providers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -878,7 +878,12 @@ const buildProvidersCached = unstable_cache(
// none of them on /products/blockmachine.
// v19: lockstep with bench-unfiltered-v98. Robinhood Chain gains its
// l1-finality appearance.
["providers-v19"],
// v20: 0x leaves benches 304 and 033, so it is measured nowhere. Two
// things move, not one: /products/0x stops existing (it is in
// REMOVED_PRODUCT_SLUGS now), and every aggregator that shares those two
// boards is ranked against a cohort one smaller, which is printed on its
// own product page and in its compare and answers copy.
["providers-v20"],
// 900 s: the provider index feeds products / compare / answers, whose
// numbers move slowly, and this revalidate is also the effective ISR
// period of those ~500 pages (Next takes the min across a route's caches).
Expand Down
8 changes: 8 additions & 0 deletions src/lib/removed-benches.ts
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,14 @@ export const REMOVED_PRODUCT_SLUGS = new Set([
// the provider in getProviders() output while the page 404s; 410 guard
// + sitemap exclusion stop the smoke-gate rollback until Redis flushes.
"goldrush",
// 0x left benches 304 and 033 on 2026-10-25, so it is measured nowhere
// and getProvider("0x") returns nothing: the page 404s instead of 410ing,
// and /products/0x was a 200 listed in the sitemap. Same shape as the
// goldrush note above and the reason it matters twice: the worker's blob
// keeps the provider in its product list for a sweep or two after the
// spec changes, so without this the sitemap keeps advertising a URL that
// has started 404ing, which is what rolled the Blast deploy back.
"0x",
// zerion and routescan appear in stale Redis results from retired benches
// (indexing-freshness for zerion, explorer-chain-coverage for routescan)
// but have no /products/<slug> page. 410 guard prevents smoke-gate failures.
Expand Down
2 changes: 1 addition & 1 deletion src/lib/spec-provider-set.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,6 @@ describe("the provider set behind the cache keys", () => {
"Then set EXPECTED to the hash in this failure, and add the provider",
`to PRODUCT_NAMES if it needs a display name. Providers now: ${count}.`,
].join("\n"),
).toBe("9ca6a160e665dcdf");
).toBe("eecad3db65110993");
});
});
16 changes: 13 additions & 3 deletions src/lib/spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,13 @@ const loadBenchmarkUnfilteredCached = unstable_cache(
// specs declare region and tier, so bench-filters moves with this one.
// v98: Robinhood Chain joins l1-finality. No dimensions there, so
// bench-filters stays.
["bench-unfiltered-v99", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
// v100: 0x leaves benches 304 and 033. A cached entry holds the provider
// set current when it was written, and the direction matters as much as
// the fact: v92's note covers a bench GAINING a row, this is a bench
// LOSING one, and a stale entry would keep drawing a declared provider
// that no longer exists in the spec. Both benches declare a chain
// dimension, so bench-filters moves with this one.
["bench-unfiltered-v100", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
{ revalidate: 300, tags: ["benchmarks"] },
);

Expand Down Expand Up @@ -769,7 +775,8 @@ const loadAllBenchmarksCached = unstable_cache(
// provider sets leave the hash, and a cached aggregate written before
// this would still list a bench that no longer exists).
// v91: lockstep with bench-unfiltered-v98 (Robinhood Chain on l1-finality).
["all-benchmarks-v92", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
// v93: lockstep with bench-unfiltered-v100 (0x leaves benches 304 and 033).
["all-benchmarks-v93", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
{ revalidate: 300, tags: ["benchmarks"] },
);
export const loadAllBenchmarks = cache(loadAllBenchmarksCached);
Expand Down Expand Up @@ -893,7 +900,10 @@ const loadBenchmarkFiltered = unstable_cache(
// declare region and tier, and BlockMachine's rows are on the Private
// access tab, which is a filtered variant: without this they are invisible
// on exactly the tab they belong to.
["bench-filters-v40", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
// v41: lockstep with bench-unfiltered-v100. Benches 304 and 033 both
// declare a chain dimension, so every per-chain variant caches its own
// copy of the provider set and a stale entry keeps 0x on each tab.
["bench-filters-v41", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
{ revalidate: 300, tags: ["benchmarks"] }
);

Expand Down
1 change: 1 addition & 0 deletions src/middleware.ts
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ export const config = {
"/alternatives/chainlink",
"/alternatives/jupiter",
"/alternatives/pyth",
"/products/0x",
"/products/blast",
"/products/bitquery",
"/products/goldrush",
Expand Down
Loading