Repository navigation
bench 201 moves to DeFiLlama, and the four benches that need Dune stand down - #2698
Merged
Merged
Conversation
…nd down The Dune team trial ends 2026-10-02 with no paid plan committed, so anything that cannot run without Dune has to stand down and anything that can run free has to move. Bench 201 moves. It published a volume figure and a fee figure, and both are free and daily on DeFiLlama: /summary/dexs/<slug>?dataType=dailyVolume for the routed volume and /summary/fees/<slug>?dataType=dailyRevenue for the app's own commission. Not dailyFees, which adds the venue's own cut and reads about twice as much. The trading-app-volume harness already had the slug mapping and the per-app latest-closed-day anchoring, so the fees leg went there rather than into a second convention, and the bench now reads its gauges. That also gives 201 an identity distinct from 267: the majors with what they charge, against the full fourteen-app cohort with per-chain splits. BasedBot leaves the table. It has no dexs adapter and no fees adapter on DeFiLlama and no published fee wallet list anywhere, so there was nothing to attribute it from under either source. Benches 203, 206, 207 and 232 are gated to staging, not removed: the harnesses and the freshness guard all work and they come back by deleting four lines. 203 goes with them because its companion harness is its own metered Dune query, so re-sourcing only its headline would leave the rest of the page dark. The hub and the /products view now take their columns from one filtered list, so a gated bench takes its column and its link with it instead of leaving a stripe of dashes and four links into 404s, and a row with no figure at all is not listed. The copy, the table sort, the headline KPI and the OG image no longer promise wallet counts or trade sizes on a deployment that cannot serve them. Staging still shows all seven columns.
… spec Bench 201's three take panels declared higher_is_better while the hub hardcoded the opposite and told the reader "lower is cheaper for the trader". With Axiom near 1% and another app at half that, the bench page ranked Axiom first and the hub painted the cheaper app green. A lower cut is the one that is true for a reader, so all three panels now say so. The hub no longer keeps its own copy: a panel-backed column takes its direction from the panel, the matrix reports the resolved direction, and the headline KPI reads that rather than the literal. The spec is the single place that decides which end of a column is better.
…g the row The volume leg already promised that one DeFiLlama hiccup never blanks a row. The fees leg had none of that: a single timeout or 429 was logged and dropped, which deleted the commission and take-rate gauges for the full hour to the next tick. On production those two are the only figures Axiom, Trojan and Photon have, so the rows left the page and the platform count fell with them, cached for five minutes. Nothing counted the error and health stayed 1. The previous sweep's commission is now carried forward per day, and the error increments trading_app_fetch_errors_total so it is visible. The commission windows also stop being anchored on the volume day. A fees adapter can run a day behind the dexs one, which the code already expected, and anchoring on the volume day deleted the 1d gauges for as long as that lasted. They now end on the latest day that has both legs, that day is published as trading_app_revenue_day_unix, and the walk back is bounded by the same 72 hours the volume leg uses for a stale adapter. Bloom's fees adapter last published on 2026-09-15: without the bound it would have shown a twelve-day-old cut labelled as the latest, so it publishes no commission at all instead.
…rom prose The SOL badge was inferred by matching a spec formula for the word "solana" with no "+" beside it. GMGN's new formula reads "sum of every chain's daily routed volume, most of it outside Solana", so the hub badged its commission and take rate "Solana only" while the harness summed ten chains and the same page's footer said so. A column now declares its breadth, and no cell carries a badge inferred from text. Where an app's figures are narrower than its neighbours' the row's own Chains cell says so, which is measured from the adapter rather than read out of a sentence. scopeFromFormula is gone.
Bench 205 is not gated, so it ships to production alongside 201, and it said two things that are now wrong. It named Mobula and a Dune community dataset as bench 201's source, which 201 stopped using twice over. And it argued that DeFiLlama's revenue over volume overstates the fee rate, citing 1.2-2.9% for Axiom on Sep 15-17 2026, which is exactly the ratio bench 201 now publishes. Re-reading the same endpoints on 2026-09-27 gives 0.74%, 0.60% and 0.51% for those three days, so DeFiLlama restated them as its volume adapter decoded more venues, and the ratio now sits at or under the roughly 1% Axiom charges. The old claim does not reproduce and the pages said nothing about it. So 205 keeps its position for a different and measured reason: the ratio moves between 0.37% and 1.07% for one app across a fortnight, which is too unsteady to order a board by, and 201 publishes it as a daily figure with that caveat. The volume is still a floor by DeFiLlama's own methodology, but how much is missing is not something either bench can measure, so neither asserts it now.
The /products/<slug> block was headed "On-chain activity · Dune dataset" and footnoted "Dune figures as of", both hardcoded. On production it holds DeFiLlama commission and take rate plus an App Store rating and no Dune figure at all, so it credited the wrong source for everything on it. A bench now maps to the name of its source, the heading and the footnote are built from the columns this deployment serves, and the footnote's scope sentence points at the Chains cell rather than the SOL badge that no longer exists. The harness doc comment says bench 201 reads it, and the hub names benches 203, 206, 207 and 232 where it says which ones are paused.
… cannot drift, and a missing adapter is not an error Three from the re-review, all in the new fees leg. The carry-forward ran on every sweep, not only on a failed read. So a day the fees adapter restated to negative, or stopped returning, got its old positive value back, which is the opposite of the rule the line above it and the spec both state. It now runs only when the read failed: a read that succeeded is the current truth. The walk-back to the newest day with both legs was bounded against the volume anchor, and that anchor can itself already sit 72 hours back for a stale dexs adapter. Three more days from there put a six-day-old cut on the board while health still read 1. It is bounded against the cohort's last closed day now, so 72 hours is the whole budget, and the comment and both Help strings say that instead of implying the latest day. Several bots in the cohort have a dexs adapter and no fees one, and a 404 was counted as a fetch error on every sweep, which would bury a real outage. A 404 is an app with no adapter of that kind and comes back as an empty series. Also: the history blob's Source named only the dexs endpoint; bench 201's methodology said a stale adapter "loses its windows" when a window it can still fill to 80% coverage survives, and it now says the commission can trail the volume by up to three days; bench 267 says where it ends and 201 begins, which the PR body claimed both specs did and only 201 did; and dune-platform-volume stops calling itself bench 201, which it no longer feeds.
This was referenced Sep 27, 2026
bakabhai993
pushed a commit
to bakabhai993/OpenChainBench
that referenced
this pull request
Sep 29, 2026
Ships to production what PRs ChainBench#2697 and ChainBench#2698 landed on dev for the trading apps, and nothing else. The Dune team trial ends 2026-10-02 with no paid plan committed, so bench 201 moves onto a free source and the four benches that cannot run without Dune stand down. Bench 201 reads DeFiLlama's open dexs and fees adapters through the trading-app-volume harness: routed volume from dailyVolume, the app's own cut from dailyRevenue, and the take rate over the same days. Not dailyFees, which adds the venue's cut and reads about twice as much. BasedBot leaves the table, having no adapter of either kind and no published fee wallet list. pump.fun becomes measurable again. Benches 203, 206, 207 and 232 are gated to staging, not removed: they come back by deleting four lines the day the plan is paid for. dune-platform-volume comes along so the source they read is the one this repo describes, off the frozen third-party datasets and onto our own SQL with the freshness guard, which is why none of them ever served a stale figure as current. The hub and the /products view take their columns from one filtered list, so a gated bench takes its column and its link with it rather than leaving a stripe of dashes and four links into 404s, and no page credits Dune for a figure that came from DeFiLlama. removed-benches.ts was edited by hand, not taken from dev: this branch keeps 273, 274 and 275 gated where dev has them live, and does not carry 280 or 281, which do not exist here. The three cache keys are this branch's own numbering, v84, v77 and v11, not dev's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Dune team trial ends 2026-10-02 and no paid plan is committed. So anything that cannot run without Dune stands down, and anything that can run free moves off it. Nothing here throws away the work from #2697: the freshness guard is the reason none of these benches ever lied, and the Dune path stays in the tree for the day the plan is paid for.
1. Bench 201 moves to DeFiLlama
Bench 201 published exactly two things, a volume figure and a fee figure, and both are free and daily:
GET /summary/dexs/<slug>?dataType=dailyVolumeGET /summary/fees/<slug>?dataType=dailyRevenue, the app's own cut. NotdailyFees: that adds what the venue underneath charges. For Axiom on 2026-09-26 it reads $2.27M against $1.38M of revenue, so it is a different quantity. The methodology says which one it is and why.The fees leg went into
harnesses/trading-app-volume, which already had the DeFiLlama slug mapping and the per-app latest-closed-day anchoring, rather than into a second convention. It now publishestrading_app_revenue_usd{app,window}andtrading_app_take_rate_pct{app,window}alongside the volume gauges, and bench 201 reads them. The take rate divides commission by the volume of the same days, so a fees adapter running a day behind the volume one cannot read as a lower cut.That also gives 201 an identity distinct from 267, which had become a real question once both were cross-chain: 201 is the majors with what they charge; 267 is the full fourteen-app cohort with per-chain splits and a year of history. Both specs and the 201 FAQ say so.
Cross-check: our Dune SQL against DeFiLlama, data day 2026-09-26
Our figures are the Solana leg only (from #2697, still running); DeFiLlama's are every chain the adapter covers.
The three apps that trade mainly on Solana agree to within 5% on volume, and Axiom and Photon agree to within 2.3% on the app's cut, which is the check that matters: it says the two sources are measuring the same quantity. GMGN and FOMO read much higher on DeFiLlama for the documented reasons (ten chains, and Relay legs booked on Solana). pump.fun comes back as a measurable row, which fee-wallet attribution could not give it. BasedBot leaves the table: no dexs adapter, no fees adapter, no published wallet list, so there was nothing to attribute it from under either source, and a permanently blank row is worse than no row.
One thing to know about the source: DeFiLlama restates the most recent day for about a day. Axiom's 2026-09-26 revenue read $1,381,516 when I checked the API and $800,892 when the harness swept an hour later. The bench 267 disclaimer already said this and 201's now says it too.
2. What stands down
Added to
DEV_ONLY_BENCH_SLUGS, gated not removed, with the un-gate condition written next to them (a paid Dune plan):solana-avg-trade-size, 207solana-unique-traders, 232trading-platform-wallets— all three readdune-platform-volume, whose one execution a day is metered.memecoin-platforms— my call, and here is the reasoning. Its headline take rate readsdune-platform-volume, but its coverage and fee-paying-rate columns come from thememecoin-platformsharness, which is its own separate metered Dune query. Re-sourcing only the headline would leave the rest of the page dark. Bench 201 now carries a take rate from DeFiLlama, so the signal is not lost while 203 waits.Production keeps, on this hub: 201 (volume and commission), 267 (daily volume), solana-dex-volume, solana-launchpad-wars, app-store-ratings. 268 terminal-fill-quality stays dev-only as before.
3. The hub
The hub had three hardcoded slug lists and only one of them was filtered. Now there is one filtered source of truth:
TRADING_APP_COLUMNSinsrc/lib/trading-apps.tsfilters itself byisDevOnlyBench, so a gated bench takes its column with it instead of leaving a stripe of dashes. The commission and take rate columns arrive in place of the four that left. A column can now read a named metric panel, which is how it gets 201's commission alongside 201's volume headline.GROUPSlink grid filters the same way, so the four links into what would be 404s are gone, and a group that empties disappears.loadTradingAppMatrix(), the same loader/products/<slug>uses, so the two cannot drift.Verified both ways on a local dev server:
VERCEL_ENVunset)VERCEL_ENV=production)hrefto any gated benchSitemap,
llms.txt,llms-full.txt,/api/citable,/api/llm-context, the catalog, the category hubs and the product appearance counts all filter throughloadSpecs()or an existingisDevOnlyBenchcall, so they needed no change. No answer YAML references any of the five slugs./trading-appsstays in the sitemap unconditionally because the hub still has five live benches on it, unlike/capitalwhose whole cohort can be gated.One pre-existing thing I did not change: a direct hit on a gated bench URL returns 404 rather than 410, because
src/middleware.tsonly carriesREMOVED_BENCH_SLUGS. That is how the four already-gated benches behave, and 404 is arguably right for "not yet released", so changing it is its own decision.Verification
pnpm validate(240 specs),pnpm typecheck,pnpm lint(0 errors),bun test src/lib(328 pass),scripts/editorial-check.shon the touched spec,go vetand a build of the harness, and the two dev-server renders above.The harness is deployed on the VPS and Prometheus has the new series: 12 apps on
trading_app_take_rate_pct{window="1d"}, Axiom 0.51%, GMGN 0.79%, Photon 1.04%, Trojan 0.79%, FOMO 0.46%, Terminal 0.54%, pump.fun 0.67%. Old image taggedocb-trading-app-volume:rollback-20260927. Two apps are correctly absent from the revenue gauges: Bloom has no fees adapter, and DeFiLlama reported a negative daily revenue for Pepeboost, which the harness drops rather than publishing a negative take rate.