You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Three limitation lines that are an option, a dangling reference and a test fixture #22
Three lines of the bundled benchmarks do not do what the page's rule asks of a limitation (it must affect a choice, a required action or the reading of a result):
Terminal-Bench lists "Each task's prebuilt image is pulled by default. The image option builds it here instead..." as a limitation. It describes an option.
Terminal-Bench's upstream-settings.json says of environment.storage_mb that it is "said in the limitations". No limitation says it.
τ³-bench says "The mock domain and voice mode are not run." The mock domain is the upstream's test fixture (10 tasks for its own tests), not part of the benchmark or its leaderboard.
Wanted
The image line is under Good to know; the settings inventory no longer points at a limitation that does not exist; τ³-bench's line names voice mode only.
Acceptance
Terminal-Bench's image line is in good_to_know, not limitations.
The storage_mb row of Terminal-Bench's settings inventory does not refer to the limitations.
τ³-bench's limitation does not mention the mock domain.
Activity