Repository navigation
the widest spread on bench 284 is not the returns, it is how much they trade - #2845
Merged
Merged
Conversation
…y trade
Probing the Recall API past the two endpoints the harness used turned up
/agents/{id}, which carries lifetime trade counters and each agent's on-chain
wallet. Neither appears on the per-competition rows we were reading.
Under identical rules, same harness, same prompt, same pair, same six days,
the roster trades between 2.2 and 88.7 times a round. A forty-fold spread, and
the two halves of one model sit at opposite ends: it is the input format,
numbers against an image, that moves this far more than the model does.
Nothing anywhere publishes it, and a leaderboard of returns cannot show it.
gpt-5.2 chart 2.2 grok 4 chart 3.0
gemini 3 pro chart 6.4 opus 4.5 chart 16.4
sonnet 4.5 vision 29.6 gemini 3 pro vision 37.9
gpt-5.2 vision 48.7 grok 4 vision 88.7
## What it does to the headline, and what it does not
At the roster average of 29 trades a round and Aerodrome's 0.05 percent
volatile-pool fee, charging the full portfolio to every trade would cost about
1.45 percent a week against a measured shortfall of 1.07. On that arithmetic
friction accounts for the whole gap.
The cross-section refuses to go along with it. The correlation between trades
per round and alpha is -0.25 over the eight agents, Spearman -0.41, neither
significant at n=8. The quietest agent still gives up 16.4 points; the busiest
finishes third.
So friction is large enough to matter in aggregate and is not what separates
the agents. Both halves ship, and the page says the fee figure is an upper
bound rather than an explanation, because the arithmetic assumes full
portfolio turnover on every trade and the real cost depends on the NOTIONAL,
which this API does not serve.
That number is on-chain. The upstream publishes each agent's wallet, so every
swap is checkable on Base, and the next phase is a measurement rather than a
hunt: the eight addresses are known.
## Shipped
activity.go fetches one detail record per ranked agent, after the roster is
settled so no request is spent on an entry that will not be published.
trading_agent_trades_per_round and _trades_total, plus a visible ledger column
and a hub section.
The counters are lifetime rather than per arena, which would dilute them if
these agents ran elsewhere. They do not: checked against each agent's own
round list, 35 to 37 of every agent's 32 to 34 completed rounds are this
arena. A missing counter publishes as missing, never as zero, because zero
would read as an agent that never trades.
Plugin gains R10, which re-derives trades per round from a fresh upstream walk
and reports the observed spread, so the forty-fold figure in the copy is
checked rather than asserted.
Harness already rebuilt and running; the worker needs its rebuild after this
merges, in that order.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
the widest spread on bench 284 is not the returns, it is how much they trade
Probing the Recall API past the two endpoints the harness used turned up
/agents/{id}, which carries lifetime trade counters and each agent's on-chain
wallet. Neither appears on the per-competition rows we were reading.
Under identical rules, same harness, same prompt, same pair, same six days,
the roster trades between 2.2 and 88.7 times a round. A forty-fold spread, and
the two halves of one model sit at opposite ends: it is the input format,
numbers against an image, that moves this far more than the model does.
Nothing anywhere publishes it, and a leaderboard of returns cannot show it.
What it does to the headline, and what it does not
At the roster average of 29 trades a round and Aerodrome's 0.05 percent
volatile-pool fee, charging the full portfolio to every trade would cost about
1.45 percent a week against a measured shortfall of 1.07. On that arithmetic
friction accounts for the whole gap.
The cross-section refuses to go along with it. The correlation between trades
per round and alpha is -0.25 over the eight agents, Spearman -0.41, neither
significant at n=8. The quietest agent still gives up 16.4 points; the busiest
finishes third.
So friction is large enough to matter in aggregate and is not what separates
the agents. Both halves ship, and the page says the fee figure is an upper
bound rather than an explanation, because the arithmetic assumes full
portfolio turnover on every trade and the real cost depends on the NOTIONAL,
which this API does not serve.
That number is on-chain. The upstream publishes each agent's wallet, so every
swap is checkable on Base, and the next phase is a measurement rather than a
hunt: the eight addresses are known.
Shipped
activity.go fetches one detail record per ranked agent, after the roster is
settled so no request is spent on an entry that will not be published.
trading_agent_trades_per_round and _trades_total, plus a visible ledger column
and a hub section.
The counters are lifetime rather than per arena, which would dilute them if
these agents ran elsewhere. They do not: checked against each agent's own
round list, 35 to 37 of every agent's 32 to 34 completed rounds are this
arena. A missing counter publishes as missing, never as zero, because zero
would read as an agent that never trades.
Plugin gains R10, which re-derives trades per round from a fresh upstream walk
and reports the observed spread, so the forty-fold figure in the copy is
checked rather than asserted.
Harness already rebuilt and running; the worker needs its rebuild after this
merges, in that order.