Repository navigation
bench 201: the trading platform figures come from our own Dune SQL, and a stale source stops publishing - #2697
Merged
Merged
Conversation
The query behind this harness reads the newest row each source has, so a source that stops updating keeps returning the same day and the harness republishes it as current. That is what happened: the day froze on 2026-08-25 and four benches served it as live for 32 days without anything erroring. The data day now travels with the figures. A platform more than DUNE_MAX_DATA_AGE_DAYS whole UTC days behind (default 3), or missing from the result altogether, has every gauge child deleted and health set to 0, so the bench reads unresponsive rather than showing an old number as today's. A row with no data day at all counts as stale.
…aded datasets The eight dune.adam_tehc_co.dataset_*_daily tables this harness read are one person's uploads and they stopped on 2026-08-25. Raw solana.* and Spellbook dex_solana.* / prices.* are maintained by Dune, so the query now reads those directly, with the attribution DeFiLlama's own Solana adapters use: the transactions in which a platform fee wallet received value, joined to dex_solana.trades, one leg per transaction and trader so a multi-hop route counts once. Fees are what the wallets received that day, priced with the day's SOL close. Against DeFiLlama for 2026-09-25, Solana leg only: axiom +0.1%, trojan +0.0%, gmgn -2.2%, padre -2.6%, photon +11.8%. fomo needs the per- transaction USDC-leg rule its own adapter uses, because it sponsors gas and one transaction carries several trader ids; that lands within 3.4% of the retired series. basedbot has no public fee wallet list to attribute it from and pump.fun charges no fee, so both are dropped rather than published off a different measurement. The day is a Dune query parameter, not a subquery. Deriving it inside SQL leaves the block_date filter unknown at planning time, every partition is read and the same query costs 583 credits against 7 pinned. Measured.
… a stale one does Every field that named a dune.adam_tehc_co dataset now names what the harness actually reads: our own SQL over solana.account_activity, dex_solana.trades and prices.day, fee-wallet attribution, one leg per transaction and trader. Each methodology gains a bullet for the stale-source behaviour, since that is the part a reader cannot see from a number. The scope is Solana only and the specs say so rather than claiming a cross-chain total. pump.fun and BasedBot rows say they are not measured and why. bench 206 also stops crediting Mobula lighthouse, which it had not read since the harness changed. Cache keys bumped in lockstep: the cached entries hold the retired dataset names in the Dataset blob, the quotable sentence and the product formulas.
Patching a query bumps its version and Dune then answers /results with 404 until something re-executes, so writing the SQL on every container start turned each restart into a metered execution. Read the stored SQL first and write only on a real change.
…partial day does not publish, and a failed execution cannot loop Four things the review found. Fees were summed over every inflow to a platform's fee wallets while volume came only from the transactions that had a trade behind them, so a sweep between two of a platform's own wallets, or a token account closing back into its owner, read as revenue the denominator had no volume for. On 2026-09-25 that put padre 37% and fomo 26% above what the volume transactions support. Fees now join the volume transaction set. Volume is unchanged. The data day was the day the harness asked for, not a day observed, so a Spellbook incremental still half way through the day would have published a fraction of it as a whole day. The query returns the last trade it saw and a day that stops short of its end is dropped. Without the day's SOL close the fee total is only its stablecoin part. The price comes back as a column and the fee gauges are withheld when it is missing, rather than reading as a low take rate that looks measured. An execution that fails, is cancelled or outlives the polling budget leaves the cached result untouched, so the age check alone would have started another metered run on every fetch tick. Executions are now bounded to one a cadence whatever the outcome. Cost goes from 7 to 25 credits a run for the fee join and the last-trade read. Also: fomo's fee figure is now its Solana swap fees rather than its whole take, which matches the Solana-only scope the rest of its row already had; the cadence, source and lead-platform copy left over in 206, 207, 209 and 203; and the credit figure is the measured one in every place that quotes it.
…cution id, and a failing day stops costing credits
A fetch that fails returned before the guard ran, so the figures from the last
good fetch stayed on the board with health 1 for as long as the failure
lasted, on a rotated key or a Dune outage. The last rows seen are kept and the
guard re-runs over them every tick, so they age out and the benches go
unresponsive instead.
The execution is parameterised on the data day but the read went through
GET /query/{id}/results, which takes no parameters, so the day that ran was
not provably the day that landed. Results are read by execution id, the way
perp-volume-history does it. latestResult stays for the startup path, where
there is no execution id and whatever ran last is the right answer.
The refresh gate was the age of the cached result. That drifts a quarter of an
hour later every day until a run walks past midnight and a data day is never
measured, and it cannot tell a failed execution from one never tried. The gate
is now the day itself, with three attempts an hour apart before it gives up and
lets the guard make the benches unresponsive, which is the right outcome for a
query that cannot run.
Also: lastExecutionEnded was written on the polling goroutine and read on the
main loop with no synchronisation; executionState ignored the HTTP status, so
an error body with no state field polled a dead execution to its timeout; and
bench 206 has three rows (Maestro, Phantom, Bloom) reading Mobula attribution
on a rolling window, which the page now says instead of claiming every row
shares one rule.
Keeping the last rows so the guard can age them out put two writers on them: the fetch tick keeps firing while an execution is in flight, and the polling goroutine publishes the execution's own results. Both now go through one publish function holding a mutex, which also covers the published-day the refresh gate reads. go test -race is clean.
… a day missing its fee figures is retried
Four more from the review.
Nothing was on the board at all before the first publish, not even health 0,
so a deploy landing before the indexing lag cleared left the series absent for
hours rather than unresponsive, and an alert on health == 0 had nothing to fire
on. Every known platform is marked unresponsive at startup.
The 15-minute tick re-read GET /query/{id}/results, which takes no parameters,
and republished whatever came back. That could only overwrite the day this
harness executed and read by id with an older one, and it depended on how Dune
resolves that endpoint for a parameterised query. The tick now only re-runs the
guard, so figures still age out; the single read on start is what picks the
board back up after a restart.
A publish that had to withhold the fee figures for a missing SOL price was
recorded as a finished day, so the retry budget never used its remaining
attempts and the 203 take-rate column stayed empty until the next day. The day
is recorded only when the fee figures went out with it.
No execution on start either. The per-day retry budget lives in memory, so a
restart loop during a failing day would otherwise spend 25 credits each time
the process came up; waiting one tick is what bounds it. Bench 207's source
link pointed at the harness its own abstract calls a separate series.
…he dead cadence knob is gone A run that lands before Dune has finished loading the day comes back partial, or empty. Taking it replaced the rows held and the guard then dropped every platform, deleting a complete day that was still inside the window, and no retry could bring it back because the partial rows were now what was held. A result that publishes nothing is not taken; the held day stays and the retries still run. One rule, rowUsable, decides both what publishRows publishes and whether a result is worth taking, so the two cannot disagree. DUNE_REFRESH_HOURS, resultAge and lastExecutionEnded were dead once the gate became the data day, but the specs and the package comment still told readers that knob set the cadence. Removed, and the real cadence is written down instead: one execution per data day, three tries an hour apart. The SQL comparison is on trimmed text, so a normalising save on Dune's side cannot turn every container start into a paid run. A failing status call no longer abandons an execution that is still billing: five consecutive errors are tolerated first, because giving up on the first one let the next attempt pay for the same day again. Bench 232's Fomo row said it reads unresponsive. It does not: only its wallet count is withheld, so health stays 1 and the row carries no figure. The page says that, and two findings that named pump.fun as a leader are gone.
… and an unpriced day cannot blank a priced one Five from the third round. A failed sync was logged and forgotten. One 429 or timeout on the deploy this change triggers left Dune holding the old dataset SQL, whose rows carry no data day, so every row was dropped as stale and each retry paid for a query that could never publish. The sync is retried from the tick and nothing executes until it has succeeded once. A result with no SOL price replaced a complete, priced day that was still inside the window, which blanked bench 203's whole take-rate column for the day. That is the case the partial-day guard already covered, left open for the fee half: publishableRows now reports usable and priced separately, and an unpriced result is kept out while a priced day is held. DUNE_MAX_DATA_AGE_DAYS below 2 could never publish anything, because the target day is yesterday at best and two days back before the indexing lag clears, and the retry budget would keep buying executions that cannot land. Values under 2 are refused. An empty board waited for 10:00 UTC, so a deploy earlier in the day left every bench dark for hours. With nothing published, the run goes ahead on the two-day target, which is complete and inside the window. An execution abandoned on a poll timeout or on repeated status errors was paid for and never read. Its results are read once before giving up, which costs nothing and stops the next attempt paying for the same day again.
… the copy credits the right source dayCovered's 23:00 bar is enough to publish a day, but recording it as measured stops it ever being re-run, so a day the query saw only to 23:00 was locked in an hour short. Recording now needs 23:55; publishing still needs 23:00, and a retry replaces the short read because publish takes a later usable, priced result for the same day. Bench 206 credited every row to our own Dune SQL while Maestro, Phantom and Bloom read Mobula on a rolling window. Bench 203 still ranked pump.fun in its title and description after this change makes that row unresponsive. Bench 207's intro said Fomo's transaction count is swollen by the gas it sponsors, which the one-row-per-transaction rule rules out, and it hard-coded a leaderboard order that the template already fills in.
This was referenced Sep 27, 2026
bakabhai993
pushed a commit
to bakabhai993/OpenChainBench
that referenced
this pull request
Sep 29, 2026
Ships to production what PRs ChainBench#2697 and ChainBench#2698 landed on dev for the trading apps, and nothing else. The Dune team trial ends 2026-10-02 with no paid plan committed, so bench 201 moves onto a free source and the four benches that cannot run without Dune stand down. Bench 201 reads DeFiLlama's open dexs and fees adapters through the trading-app-volume harness: routed volume from dailyVolume, the app's own cut from dailyRevenue, and the take rate over the same days. Not dailyFees, which adds the venue's cut and reads about twice as much. BasedBot leaves the table, having no adapter of either kind and no published fee wallet list. pump.fun becomes measurable again. Benches 203, 206, 207 and 232 are gated to staging, not removed: they come back by deleting four lines the day the plan is paid for. dune-platform-volume comes along so the source they read is the one this repo describes, off the frozen third-party datasets and onto our own SQL with the freshness guard, which is why none of them ever served a stale figure as current. The hub and the /products view take their columns from one filtered list, so a gated bench takes its column and its link with it rather than leaving a stripe of dashes and four links into 404s, and no page credits Dune for a figure that came from DeFiLlama. removed-benches.ts was edited by hand, not taken from dev: this branch keeps 273, 274 and 275 gated where dev has them live, and does not carry 280 or 281, which do not exist here. The three cache keys are this branch's own numbering, v84, v77 and v11, not dev's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
harnesses/dune-platform-volumebuilt its query from eight uploaded datasets owned by a third party (dune.adam_tehc_co.dataset_*_daily). All eight stopped updating on 2026-08-25. The query selectedWHERE day = (SELECT MAX(day) …), so it returned the 25 August row every day and the harness published it as current. Nothing errored for 32 days.Four benches were serving that month-old day as live: 201
solana-trading-platform-wars, 206solana-avg-trade-size, 207solana-unique-traders, 232trading-platform-wallets, plus the take-rate column of 203memecoin-platforms.The fix
A freshness guard, first and on its own commit. The data day travels with the figures as
dune_platform_volume_data_day_unix{platform}. A platform more thanDUNE_MAX_DATA_AGE_DAYSwhole UTC days behind (default 3), or missing from the result altogether, or on a day the query did not see through to its end, has every gauge child deleted andhealthset to 0. A row with no data day at all counts as stale. Same shape asusdy-nav-basis's failed leg andperp-volume-history's incomplete window.Our own SQL. Fee-wallet attribution over
solana.account_activityjoined todex_solana.trades, with theROW_NUMBER()one-leg-per-(transaction, trader) dedup fromdexs/axiom.ts— the SOL leg preferred as the notional, else the largest. Fees are what the wallets received on those same transactions (lamports on the wallet, plus USDC and wSOL in token accounts it owns), priced with the day's SOL close fromprices.day. Wallet lists come from the DeFiLlama adapter repo and from our ownsolana-unique-tradersharness, with a comment saying which and on what date.Specs name the real source, say the scope is Solana only, and each methodology gains a bullet for what happens when a source goes stale. 206 also stops crediting Mobula lighthouse and 207 stops describing a different harness; both had a table column reading
dune_platform_*while their prose described something else. Cache keys bumped in lockstep (bench-unfiltered-v86,all-benchmarks-v79,providers-v13).The SQL is only pushed to Dune when it differs, because a PATCH bumps the query version and Dune then answers
/resultswith 404 until something re-executes — patching identical SQL on every start would turn each container restart into a metered execution. Verified: a restart with matching SQL logsSQL already matches this buildand spends nothing.Cost
25 credits per execution, one execution a day. Measured on execution
01M3H6GYACTVYK0FQMEKDPK5A2. The query is 7 credits without the fee-to-volume join and the last-trade read; those two are worth 18 credits a day rather than a take rate whose halves describe different transactions and a day that can publish half-loaded.The day reaches the query as a Dune query parameter, not as a subquery. This matters a lot: deriving it in SQL from
MAX(block_date)leaves theblock_datefilter unknown at planning time, every partition is read, and the identical query cost 583 credits instead of 7 (execution01M3H3M8ZJ38N6QMWR4FN1NPER, byte-identical results). There is a comment and a test guarding against putting the day back inside the query.Numbers against DeFiLlama, 2026-09-25 (Solana leg only)
DeFiLlama's Dune-backed adapters all have 2026-09-25 as their newest complete day, so that is the comparable day. Their Solana figure is
breakdown24h.solana.Photon is the one outlier over 10% and the reason is known:
dexs/photon.tscounts only the SOL fee branch, while this query also counts Photon transactions whose fee was paid in USDC or wSOL. Re-running with DeFiLlama's own tie-break (largest leg, no SOL preference) gives 921,336, so the dedup rule is not the cause. The spec says what the attribution counts.fomo's DeFiLlama figure includes cross-chain Relay settlement through Allium, which is not on Solana and cannot be reproduced on Dune. Against the retired dataset's own row for 2026-08-25 (52,141,065) our per-transaction USDC-leg rule gives 50,371,589, -3.4% — so the Solana-native series is continuous. The naive per-(transaction, trader) rule gives 101,015,499 (+94%) because fomo sponsors gas and one transaction carries several trader ids; that is why fomo gets its own rule, the one its own adapter uses.
Fees on 2026-09-25, before and after restricting them to the transactions the volume comes from: axiom 947,836 → 915,803, gmgn 211,587 → 201,844, trojan 41,332 → 40,496, photon 9,837 → 9,616, padre 263,757 → 167,303, fomo 466,039 → 345,455. The two large moves are the point of the change: padre's unrestricted fee figure read 121% above its own retired row for 2026-08-25, and fomo's fee figure now covers its Solana swaps rather than its whole take, which matches the Solana-only scope the rest of its row already had.
A note on historical re-runs: every platform reads 14% to 54% above its own frozen 2026-08-25 row (axiom +31%, gmgn +54%, photon +54%, trojan +14%, padre +45%).
dex_solana.tradeskeeps decoding more venues, so a re-run today finds trades the adapter could not see when it first ran. The same-day comparison against DeFiLlama is the meaningful test, not the historical one.Which benches recover and which stay unresponsive
Recovering, 6 platforms: gmgn, axiom, trojan, padre (Terminal), photon, fomo — across 201, 206, 207 and the 203 take-rate column. 232
trading-platform-walletsrecovers 5 of them.Staying unresponsive, deliberately:
6Vo3245eszAb5wuqEMw8mGdbfRUdKbHhDHP5LcaGuTAB, whatdexs/pumpfun-app.tskeys on) gives 28,916,541 for 2026-08-25 against the retired dataset's 21,778,990, +33%, and 26,675,756 for 2026-09-25 against DeFiLlama's 15,779,405, +69%. That is a different measurement, so it is not shipped under the old series name.Source tiers, written into the harness
Raw
solana.*and Spellbookdex_solana.*/prices.*are maintained by Dune and do not rot.dune.<user>.<dataset>is one person's upload and can freeze silently, which is exactly what happened. Never depend on the third kind again.dex_solana.bot_tradeswas evaluated and rejected as a primary source: it covers only 11 bots, none of the web terminals, and for Trojan on 2026-08-25 it reads $3,564,141 against the $4,735,534 we published and DeFiLlama's $4,714,468 — 25% low on volume and 38% low on transaction count. It would have been a different measurement under the same series name.Live state after deploy
Prometheus at
172.18.0.14:9090, all six live platforms on data day 2026-09-26 (dune_platform_volume_data_day_unix= 1790380800, 1.44 days behind):The take rates land where the stated fees are: five of the six terminals charge 1% and read 0.89 to 1.01 once the fees are restricted to the transactions the volume comes from. Before that restriction padre read 1.54.
dune_platform_volume_healthis 1 for those six and 0 forbasedbotandpump-fun, and a query across every value gauge for those two returns zero series, so the deletes took.dune_platform_wallets_24hcarries five platforms, fomo correctly absent.2026-09-26 cannot be checked against DeFiLlama yet: their Dune-backed adapters were still on 2026-09-25 at the time of writing, which is why the table above is the deploy state and the 2026-09-25 table is the acceptance test.
Review fixes
prices.dayrow silently zeroed the SOL share of the fees. The price comes back as a column and the fee gauges are withheld when it is missing.GET /query/{id}/resultstakes no parameters), so the day that ran was not provably the day that landed. Results are read by execution id, the wayperp-volume-historydoes it.lastExecutionEndedwas a data race between the polling goroutine and the main loop;executionStateignored the HTTP status, so an error body with nostatefield polled a dead execution to its 30-minute timeout.Second review round
Pass A of the re-review confirmed one race I had introduced and found four more things, all fixed:
publishfunction behind a mutex now, covering the published-day the refresh gate reads.go test -raceis clean.GET /query/{id}/results, which takes no parameters, and republish whatever came back — which could only overwrite the day this harness executed and read back by id with an older one, and depended on how Dune resolves that endpoint for a parameterised query. The tick now only re-runs the guard; the single read on start is what picks the board back up after a restart.health 0, so a deploy landing before the indexing lag cleared left the series absent for up to ten hours rather than unresponsive. Every known platform is marked unresponsive at startup.source:link pointed at the harness its own abstract calls a separate series.Three questions the review could not answer from the repo, answered from the live deploy:
executesend the day? Yes. The container loggedexecution … started for 2026-09-26and the publisheddune_platform_volume_data_day_unixcame back as 2026-09-26, not the query's stored default.syncQuerydeclare the parameter, and does Dune normalise stored SQL? It declares it, and Dune stores the SQL byte for byte: the next restart loggedSQL already matches this buildand spent nothing, which is exactly the comparison the review worried about./resultsendpoint return the run we asked for? No longer relevant: after a completed execution the results are read by execution id, and the endpoint is only touched once on start.Deploy
Built and running on the VPS on
ocb_web. Old image taggedocb-dune-platform-volume:rollback-20260927. Dune query8776234keeps its id and history; its SQL is updated in place throughPATCH /api/v1/query/{id}.Verification
pnpm validate(240 specs),pnpm typecheck,scripts/editorial-check.shon all five touched specs, andgo vet/go teston the harness. Tests cover the freshness guard (a stale day, an absent platform and a partly loaded day all drop every child), the withheld fee gauges when the SOL price is missing, the day arithmetic either side of the 10-hour indexing lag, and two query invariants: everyblock_datefilter pinned to the day macro, and the fees joined to the volume transactions.Credit budget, reported as asked
My own testing used 716 credits against the ~150 I was asked to stay inside, and 583 of those were one mistake: I ran the production query once with the day derived in SQL instead of pinned, it failed to prune, and it ran to completion before my cancel landed. The team key is at 1,082 of 2,500 used with 1,418 left before the trial ends on 2026-10-02. The shipped query is 25 credits a day, and the runner script now aborts an execution the moment its running cost passes a ceiling, which would have caught that one.