Repository navigation
cost benches audit round: one ranking rule, the size in /api/stat, calculators off every bench page - #3433
Merged
Conversation
Flotapponnier
force-pushed
the
fix/cost-bench-audit-round
branch
from
October 30, 2026 17:34
eab618e to
8fb65a9
Compare
…lculators off every bench page Fixes from the 2026-10-30 re-audits of benches 309 and 310 (data and SEO passes), applied to bench 312 as well since it shares the code. Ranks. Product pages ranked only the prices that hold while the ledger and the calculator rank every priced row except an "at least" price at or under the leader, so /products/portal read "#8 of 8" in its bench list and "#6 of 6" in the table under it, and Privy "#7 of 8" beside "not ranked". src/lib/cost-rank.ts now holds the one rule; the product tables and the product metadata use it, with the same denominator. Machine surfaces. /api/stat ignored ?bucket=, so ?kind=trading&bucket=100k answered with the 10k cell while echoing only the kind; it now applies the size. /api/stat and the MCP rankings carry priceReading ("at most", "at least"), so Privy's floor is no longer a plain $6,500 in JSON. The variant API copies the headline seoTitle and seoDescription instead of resolving them on a filtered view under the headline's label. A new spec field, headline_slice, replaces "measured on AI agents, 50 transactions each, Medium size" in the citable headline with what the cell is. The Dataset JSON-LD of a bench with ledger_reliability: false no longer lists sample_size or percentile axes. Weight. Every bench page downloaded the cost calculators and their curves (68 KB gzip) because the shared route imported them statically; they now load through next/dynamic from a client wrapper, SSR kept. Display. The calculators print cents when a bill has cents (Dfns $2,916.67, not $2,917); product appearance rows print a published price whole ($51,299, not $51.3K); a $0 floor reads "$0 on its published rates" instead of "at least $0". Vendors priced only by the pricing benches get an H1 that says pricing, and one on several of them (Circle, Turnkey, Openfort, ZeroDev) a title naming every kind and a description from the first headline bill it has, instead of bench 309's alone. Copy. 309: Portal's meter in the Units bullet and the transactions FAQ, the changelog sentences out of the Dynamic finding and the cohort, the false Openfort cross-reference dropped, Privy's 100k bills read from the curve, the freshness gap stated. 310: the bot FAQ and finding name the curve's winners (Magic ties Circle at 10,000 transactions), Dfns and Crossmint in Units, Circle's chain scope and USDC rebate, bot and payout cells in the intro, a title that does not promise one winner. Validator. A question asking for the cheapest must be answered by a rank cell in its first sentence, and only a rank cell exempts a winner sentence (any vendor cell did). Proved by putting the old bot answer back: it fails. Harnesses (309, 310, 312). The site curves are re-read every 5 minutes and the mismatch alert fires after 15 (was hourly and 2h, blind to the 30 minutes production priced from two catalogues on 2026-10-30); the release order is in the wallet-cost README. Cache keys bench-unfiltered-v132, all-benchmarks-v125, bench-filters-v70.
Flotapponnier
force-pushed
the
fix/cost-bench-audit-round
branch
from
October 30, 2026 18:05
8fb65a9 to
0e30978
Compare
Flotapponnier
added a commit
that referenced
this pull request
Oct 30, 2026
…3434) * a kind or bucket view names itself in /api/stat and the MCP headline /api/stat/server-wallet-cost?kind=bot&bucket=large answered "Circle posts the lowest monthly cost for 10,000 AI agents at $0": the bot's figure under the metric of the headline cell, since the metric and the slice clause both describe the default view. Found while checking #3433 on staging; the old "measured on <first dimension labels>" clause had the same defect. A request with a kind or bucket filter now gets a sentence that names the view from the tab labels ("Circle posts the lowest figure on the Trading bot, 20 wallets, Large size view at $0 across 9 ranked vendors, from published list prices"), in /api/stat (headline and quote) and in the MCP get_benchmark headline. The unfiltered sentence is unchanged. No cache key: the sentence is built per request from the cached record. * product prose prints a published price whole and keeps a $0 price The intro of /products/portal read "$51.3K latest" above a table saying $51,299, and a vendor whose price is $0 (Circle for a 20-wallet bot) was dropped from it as if unmeasured. Pricing benches now print the bill whole and keep $0, and a pricing-only vendor is "priced across" its benches, not "measured".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes from the 2026-10-30 re-audits of benches 309 and 310 (data and SEO passes). They also apply to 312, which shares the code.
Ranks
src/lib/cost-rank.tsnow holds the single ranking rule. Product tables and product metadata use it, with the same denominator as the calculator and the ledger. Before,/products/portalshowed "bench №004 metadata coverage #8 of 8" in one place and "decentralized hosting model #6 of 6" in another.APIs
/api/statnow honours?bucket=.priceReadingis carried into the/api/statand MCP rankings.seoTitleandseoDescription.headline_slicespec field: the citable headline names its cell, not "measured on … Medium size".sample_sizeor percentile axes in the Dataset JSON-LD of unsampled benches.Weight
next/dynamicfrom a client wrapper, with SSR kept. Before, every bench page downloaded them (68 KB gzip).Display
Copy
Validator
Harnesses (309, 310, 312)
Not in this PR
explain()picks the reason by cap. This is latent and correct today.Checks:
bun test src/lib760 pass;tscclean;pnpm validategreen;go testgreen on the three harnesses.Cache keys: bench-unfiltered-v132, all-benchmarks-v125, bench-filters-v70.