Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
PyTorch implementation of our ECCV 2022 paper "Rethinking Confidence Calibration for Failure Prediction"
A decision-safety lab for loan approval: trains a baseline classifier, calibrates probabilities (ECE/Brier), sweeps confidence thresholds to build a coverage, quality frontier and outputs a defensible abstention policy (auto-decide vs review). Includes a Streamlit dashboard for report cards, triage UI, and data quality checks.
Decision-safe evaluation + Streamlit dashboard for AI vs Human vs Post-Edited AI text detection. Generates a reliability report card (Accuracy, Macro F1, ECE, Brier), calibration plots, confidence histograms, and a coverage-vs-performance abstention curve. Recommends an operating threshold for human-review routing.
Code for "Budgeted Classification with Rejection: An Evolutionary Method with Multiple Objectives", at IEEE CEC 2022
Robust Selective Classification of Skin Lesions with Asymmetric Costs
Reliable and Efficient Selective Machine Learning
More human oversight can make an AI agent less safe. Headroom is a human-in-the-loop firewall for coding agents that measures when to trust the human: oversight as resource allocation, not just classification.
‘‘I know that I don’t know... and I explain why’’ Robust abstention via counterfactual explanations
Event-level risk-controlled triage for conjunction data message streams
Recorded motor-imagery EEG demo with participant-disjoint evaluation, confidence-based abstention, ONNX inference and reproducible teaching exercises.
Étude académique : classification histologique, calibration des probabilités et abstention sur PathMNIST. Scikit-learn, PyTorch et Streamlit.
Risk-aware banking support router for 77 intents using calibrated classification, selective prediction, human escalation policies, and FastAPI.
Missingness-aware abstention for selective classification under MCAR/MAR/MNAR label missingness.
GenPark AI Agent Skill - Decomposes total prediction uncertainty into epistemic (model knowledge void) vs aleatoric (inherent task ambiguity) components via Shannon mutual information.
An on-device decision model for selection and preference scenarios: the training recipe, the evaluation protocol and the serving contract.
Learning When to Ask: trust calibration for agentic tool use as GP classification with a reject option. An allow/ask/block gateway learned from supervisor feedback, extended to opaque LLM judges. Paper, simulations, Lean 4 proofs and a Go harness hook.
Calibrated refusal gate for a complex holographic memory (SYNTHETIC, AGPL-3.0)
Certified document tagging under source duplication: pre-registered study on MultiEURLEX + EUR-Lex citation graph
Epiconcept R&D Internship: 16-bit Mammography Vision (GMIC AUC 0.889), Hoeffding Selective Triage (SGR) & Grounded Clinical LLMs (CATH-LLM F1 0.91)
To associate your repository with the selective-classification topic, visit your repo's landing page and select "manage topics."