sequential-testing

Runs ~20 options strategies against live market data in shadow mode, records every hypothetical fill under worst/base/optimistic assumptions, and grades each with anytime-valid e-processes. Places no orders.

Which model is really behind your API relay or agent IDE? Behavioral fingerprinting + anytime-valid sequential tests (FPR<=1%). LLMs can't be random - measured on 9 frontier models.

Sequential testing for efficient and reliable comparison of stochastic algorithms.

Sequential Hypothesis Testing with e-Values and p-Values

Calibration-driven A/B testing toolkit: SRM, CUPED/CUPAC, delta-method ratio metrics, sequential and Bayesian methods, each validated by tests.

Measure your agent harness, find where it wastes the model, and prove the fix worked. Harness-agnostic, agent-agnostic, zero dependencies. Reference implementation of HTP-1.

Uplift targeting plus an evidence-grounded local LLM agent and always-valid experiments — a growth system built to say "not you" and "not worth it". Runs offline on a laptop, synthetic data. 9-min demo: https://youtu.be/RgPZ0DGvoOw

A/B testing and causal inference scored against known answers: simulations where I set the effect, and a randomised benchmark the observational methods have to recover.

Sequential A/B Testing Framework in Python

A/B testing engine: frequentist + Bayesian + CUPED variance reduction + peeking-safe sequential testing

A/B test analysis that is correct by default, informative by design, and composable by construction. 88 classes, 60+ paper references.

Detect when a coding-agent harness or model update makes it worse or pricier on your own tasks. Pre-alpha.

Your A/B test winner is about half as good as it looked. The winner's curse, peeking and power measured on 32,363 real randomised experiments -- with a live demo that re-splits the traffic in your browser.

Anytime valid inference in Python: confidence sequences, sequential tests and e-values

Ships ML models in stages - shadow, then 1/5/25/50% of traffic - and rolls a bad one back automatically. The guardrails stay valid under constant checking: 0.6% false rollback with two identical models, where a repeatedly-read A/B test acts wrongly 36.7% of the time.

Code, data and LaTeX sources of "What can a handful of initial conditions certify?" (few-cluster inference for robot policy evaluation)

How much does peeking at an A/B test inflate false positives, and which fix works best? Simulations + tests.

Automated canary control plane for LLM configs: shadow testing, sequential inference (SPRT/CUSUM), drift monitoring

Causal bandit orchestration platform for real-time adaptive experimentation, sequential testing, and interference-aware decisioning.

Разбор A/B-эксперимента на Criteo Uplift v2.1, около 14 млн пользователей. Мощность до анализа, проверка рандомизации, CUPED, подглядывание и последовательный тест, uplift