You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Runs ~20 options strategies against live market data in shadow mode, records every hypothetical fill under worst/base/optimistic assumptions, and grades each with anytime-valid e-processes. Places no orders.
Measure your agent harness, find where it wastes the model, and prove the fix worked. Harness-agnostic, agent-agnostic, zero dependencies. Reference implementation of HTP-1.
Certified pairwise leaderboards for retrieval evaluation. WITNESS samples a few fresh human labels at known probabilities and uses official qrels and LLM judges only as variance-reducing predictors: unbiased for any judgment source, false-discovery-rate controlled, with label-budget forecasts. One-command reproduction.
Benchmark for statistically valid AI scientist systems, using audit-closed protocols, transparency logs, and sequential inference to prevent false discoveries in autonomous research agents.
Publicly verifiable proof that an LLM endpoint actually spent the compute you paid for — zero provider cooperation. Closes the effort gap (Hollow-LLM) and the transferability gap (IRIS).
Label-free drift attribution (model / noise / world / annotator) with anytime-valid guarantees and a frozen pre-registration — closes the zero-label misattribution gap 0.50→0.00 on the recoverable cause; negatives reported as measured boundaries.
Code and data for the letter "Anytime-Valid Alarms for BGP Incident Detection via Conformal e-CUSUM Monitors": online conformal p-values and betting-based e-CUSUM monitors for BGP incident alarms, with seven incidents, ordinary-day baselines and a one-command reproduction.