Repository navigation
a success rate printed to two decimals off 68 observations - #2775
Merged
Merged
Conversation
Every ledger printed its success column as toFixed(2), so the four bridge execution providers all read "100.00%": two decimals, implying resolution to a hundredth of a point, from samples of 68 to 95 executions a week whose finest possible step is 1/68, about 1.5 points. The column distinguished nothing while looking authoritative, and nothing on the page let a reader tell 100% of 68 from 100% of 68,000. Decimals now follow the sample. The useful precision of a rate from n observations is log10(n) digits less the two the percentage scale already carries, capped at the two this ledger has always shown: 68 executions 100.00% -> 100% 93 executions 100.00% -> 100% 74 executions 98.65% -> 99% 2,863 probes 99.44% -> 99.44% (unchanged, it earned them) The 30-second probe benches keep what they had; only the figures that were never supported lose anything. The cell also carries what the percentage rests on. Where nothing failed it names the ceiling the sample can support, because "100%" reads as "cannot fail" and 68 observations cannot establish anything above 98.5%. Where something failed it gives the count. An unknown sample size keeps two decimals: coarsening a figure whose sample we cannot see would be asserting an imprecision we have not established. The rule lives in src/lib/sample-precision.ts so it can be tested without rendering, and because it should apply anywhere else a rate meets a sample.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while answering Aurora's Alex Shevchenko, who asks what share of bridge swaps fail and get refunded. Our answer was four providers all reading 100.00%.
The problem
Every ledger prints its success column as
toFixed(2). On the bridge execution benches that is two decimals — resolution to a hundredth of a point — from samples of 68 to 95 executions a week, whose finest possible step is 1/68, about 1.5 points.The column distinguished nothing while looking authoritative, and nothing on the page let a reader tell 100% of 68 from 100% of 68,000.
The fix
Decimals follow the sample. The useful precision of a rate from n observations is log10(n) digits, less the two the percentage scale already carries, capped at the two this ledger has always shown.
The 30-second probe benches keep what they earned; only figures that were never supported lose anything.
The cell now carries what the percentage rests on:
because "100%" reads as "cannot fail", and 68 observations cannot establish anything above 98.5%. Where something did fail, it gives the count.
An unknown sample size keeps two decimals: coarsening a figure whose sample we cannot see would be asserting an imprecision we have not established.
Why it is in lib/
So it can be tested without rendering, and because the rule should apply anywhere else a rate meets a sample. 8 tests cover the bridge case, the probe case, the climb, the unknown sample, and that 100% of 68 and 100% of 68,000 no longer read identically.
tscclean, 422 tests pass,pnpm validatepasses, lint 0 errors.Related: #2774 fixed the polling asymmetry that made Near Intents read 36.7 s against ~1 s.