Skip to content

the widest spread on bench 284 is not the returns, it is how much they trade - #2845

Merged
Flotapponnier merged 1 commit into
devfrom
feat/284-trading-activity
Oct 7, 2026
Merged

Flotapponnier merged 1 commit into
devfrom
feat/284-trading-activity

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

the widest spread on bench 284 is not the returns, it is how much they trade

Probing the Recall API past the two endpoints the harness used turned up
/agents/{id}, which carries lifetime trade counters and each agent's on-chain
wallet. Neither appears on the per-competition rows we were reading.

Under identical rules, same harness, same prompt, same pair, same six days,
the roster trades between 2.2 and 88.7 times a round. A forty-fold spread, and
the two halves of one model sit at opposite ends: it is the input format,
numbers against an image, that moves this far more than the model does.
Nothing anywhere publishes it, and a leaderboard of returns cannot show it.

gpt-5.2 chart          2.2    grok 4 chart          3.0
gemini 3 pro chart     6.4    opus 4.5 chart       16.4
sonnet 4.5 vision     29.6    gemini 3 pro vision  37.9
gpt-5.2 vision        48.7    grok 4 vision        88.7

What it does to the headline, and what it does not

At the roster average of 29 trades a round and Aerodrome's 0.05 percent
volatile-pool fee, charging the full portfolio to every trade would cost about
1.45 percent a week against a measured shortfall of 1.07. On that arithmetic
friction accounts for the whole gap.

The cross-section refuses to go along with it. The correlation between trades
per round and alpha is -0.25 over the eight agents, Spearman -0.41, neither
significant at n=8. The quietest agent still gives up 16.4 points; the busiest
finishes third.

So friction is large enough to matter in aggregate and is not what separates
the agents. Both halves ship, and the page says the fee figure is an upper
bound rather than an explanation, because the arithmetic assumes full
portfolio turnover on every trade and the real cost depends on the NOTIONAL,
which this API does not serve.

That number is on-chain. The upstream publishes each agent's wallet, so every
swap is checkable on Base, and the next phase is a measurement rather than a
hunt: the eight addresses are known.

Shipped

activity.go fetches one detail record per ranked agent, after the roster is
settled so no request is spent on an entry that will not be published.
trading_agent_trades_per_round and _trades_total, plus a visible ledger column
and a hub section.

The counters are lifetime rather than per arena, which would dilute them if
these agents ran elsewhere. They do not: checked against each agent's own
round list, 35 to 37 of every agent's 32 to 34 completed rounds are this
arena. A missing counter publishes as missing, never as zero, because zero
would read as an agent that never trades.

Plugin gains R10, which re-derives trades per round from a fresh upstream walk
and reports the observed spread, so the forty-fold figure in the copy is
checked rather than asserted.

Harness already rebuilt and running; the worker needs its rebuild after this
merges, in that order.

…y trade

Probing the Recall API past the two endpoints the harness used turned up
/agents/{id}, which carries lifetime trade counters and each agent's on-chain
wallet. Neither appears on the per-competition rows we were reading.

Under identical rules, same harness, same prompt, same pair, same six days,
the roster trades between 2.2 and 88.7 times a round. A forty-fold spread, and
the two halves of one model sit at opposite ends: it is the input format,
numbers against an image, that moves this far more than the model does.
Nothing anywhere publishes it, and a leaderboard of returns cannot show it.

    gpt-5.2 chart          2.2    grok 4 chart          3.0
    gemini 3 pro chart     6.4    opus 4.5 chart       16.4
    sonnet 4.5 vision     29.6    gemini 3 pro vision  37.9
    gpt-5.2 vision        48.7    grok 4 vision        88.7

## What it does to the headline, and what it does not

At the roster average of 29 trades a round and Aerodrome's 0.05 percent
volatile-pool fee, charging the full portfolio to every trade would cost about
1.45 percent a week against a measured shortfall of 1.07. On that arithmetic
friction accounts for the whole gap.

The cross-section refuses to go along with it. The correlation between trades
per round and alpha is -0.25 over the eight agents, Spearman -0.41, neither
significant at n=8. The quietest agent still gives up 16.4 points; the busiest
finishes third.

So friction is large enough to matter in aggregate and is not what separates
the agents. Both halves ship, and the page says the fee figure is an upper
bound rather than an explanation, because the arithmetic assumes full
portfolio turnover on every trade and the real cost depends on the NOTIONAL,
which this API does not serve.

That number is on-chain. The upstream publishes each agent's wallet, so every
swap is checkable on Base, and the next phase is a measurement rather than a
hunt: the eight addresses are known.

## Shipped

activity.go fetches one detail record per ranked agent, after the roster is
settled so no request is spent on an entry that will not be published.
trading_agent_trades_per_round and _trades_total, plus a visible ledger column
and a hub section.

The counters are lifetime rather than per arena, which would dilute them if
these agents ran elsewhere. They do not: checked against each agent's own
round list, 35 to 37 of every agent's 32 to 34 completed rounds are this
arena. A missing counter publishes as missing, never as zero, because zero
would read as an agent that never trades.

Plugin gains R10, which re-derives trades per round from a fresh upstream walk
and reports the observed spread, so the forty-fold figure in the copy is
checked rather than asserted.

Harness already rebuilt and running; the worker needs its rebuild after this
merges, in that order.
@Flotapponnier
Flotapponnier merged commit 7c37171 into dev Oct 7, 2026
2 checks passed
@Flotapponnier
Flotapponnier deleted the feat/284-trading-activity branch November 3, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant