Repository navigation
WaveOrder 4.0: five-block config schema, migrate-config, calibrate-tf - #592
Draft
talonchandler wants to merge 15 commits into
Draft
talonchandler wants to merge 15 commits into
talonchandler wants to merge 15 commits into
Conversation
…dence score Add waveorder/calibrate: named Zernike terms (Noll indexing, orthonormal, coefficients in waves), differentiable widefield fluorescence PSF/OTF synthesis from a parameter mapping (matches the shift-variant model's numerics at the field center), the exact Whittle/Gaussian negative log evidence for Gaussian noise with a Tikhonov or spectral object prior, and the calibration penalty from parameter tolerances.
Add waveorder/config: the five-block 4.0 pydantic schema (data, imaging_system, object, noise, solver + outputs and version) per the 4.0 design document, version detection, a deterministic lossless 3.x -> 4.0 migration for phase and fluorescence configurations, the inverse 4.0 -> 3.x settings conversion that selects the same code path, and a version-aware loader. The migrate/convert round trip reproduces the directly-parsed 3.x settings field for field for every configuration in the repository (unit-tested). Birefringence configurations stay 3.x.
Add 'waveorder migrate-config' (alias mc) to convert 3.x configurations to 4.0, and route every verb's config parsing through the version-aware loader: version: 4 files run through the 4.0 schema, 3.x files run as before while printing their 4.0 equivalent once with a deprecation warning. End-to-end test asserts a migrated configuration reproduces the 3.x reconstruction bit for bit. bench.py forwards the expanded suite's per-case CLI selection when present.
Add 'wo calibrate-tf' (design section 3): free pupil_aberrations terms in a 4.0 configuration are estimated with the existing patchwise Zernike recovery (shift_invariant = one full-FOV tile, design case 5; tiled uses the configured grid, case 17; score: sharpness with a single gradient stage). Results land in tf.zarr with parameters, parameters_covariance, parameter_names, tile_grid, and provenance; shift-invariant stores also carry the synthesized transfer function. Scores other than sharpness, numeric tolerances, and grid/random stages raise NotImplementedError pending design step 4.
…arch Add the MAPk search of design step 4 for shift-invariant fluorescence: score: evidence (exact Whittle marginal likelihood) or gcv (closed form for the Wiener filter), the calibration penalty from numeric tolerances, grid / random / gradient stages chained per the config, bounds clamping, Laplace parameters_covariance, score_trace and score_grid outputs, and noise.gaussian.sigma: estimate / object.weight: auto via joint maximization with a coarse (sigma, lambda) profile before each gradient stage. calibrate-tf dispatches on score; sharpness keeps the existing recovery path.
waveorder.config.schema imports the Zernike names from waveorder.calibrate, whose package init eagerly imported the config-dependent search modules, so a cold process importing waveorder.config first crashed with a circular import (pytest's import order hid it; every regression case failed). Load run/evidence_search lazily via PEP 562 and add fresh-interpreter import tests for both entry orders.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## wo-4.0 #592 +/- ##
==========================================
+ Coverage 56.75% 61.45% +4.69%
==========================================
Files 77 90 +13
Lines 13074 14192 +1118
==========================================
+ Hits 7420 8721 +1301
+ Misses 5654 5471 -183 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add prior: {beads: {diameter}} (design example 4.3): sparse spheres at
unknown subpixel positions fit jointly with the free pupil-aberration
terms by MAP in Fourier space (analytic sphere form factor, phase-ramp
positions), with local-maximum initialization and Laplace covariance.
The known object breaks the global sign-flip twin, so bead calibration
recovers signed coefficients.
calibrate-tf now accepts -t (a tf.zarr whose parameters move the prior
center and whose non-zero covariance replaces the configured tolerances
as a correlated Gaussian prior; design section 3) and --init (start
point only). Zernike docstrings now carry a prominent waves <-> radians
conversion warning (cross-track currency is radians = 2 pi x waves).
A version: 4 configuration with a simulate: {seed, shape} block
describes the true imaging system and writes data.zarr, object.zarr and
tf.zarr (drawn parameters, zero covariance, true transfer function,
provenance: simulated) into the output directory. Implemented:
fluorescence contrast, gaussian_spectrum object source, gaussian
(sigma | snr_db) and poisson (photons) noise; a numeric tolerance draws
the truth from its prior under the seed. 3.x configurations (and 4.0
without a simulate block) keep the existing phantom simulation.
Closure test: evidence calibration on simulated data recovers the drawn
coefficient.
- wo reconstruct on a 4.0 config with free parameters but no
solver.calibration block now fails loudly instead of silently running
the 3.x sharpness optimizer (the 4.0 default score is evidence, which
runs through calibrate-tf); an explicit sharpness block still selects
the 3.x path.
- Bead detection thresholds are configurable (object.prior.beads
{snr_threshold, max_beads, min_separation}) and relax adaptively (with
a warning) before failing with actionable advice.
- Design 4.5 validations added: beads-requires-free-parameter at schema
level, grid point-count warning (shorthand expanded to its Zernike
terms), tiled all-free sharpness overfit warning.
- -t prior: singular covariance blocks are rejected with a clear error;
grid stage ranges center on the -t-moved prior center rather than the
config nominal.
- Documented: the evidence path drops the DC term (exclude_dc) and
parameters_covariance is a profile (sigma, lambda fixed) curvature; the
padding z default 0 deviation from the design text; fluorescence_psf
focal-plane wording; calibrate-tf prints which time index and channel
it uses; detect_version rejects empty YAML files with a clear message.
Each case's 3.x config is converted with 'wo migrate-config' and the reconstruction runs from the 4.0 file, so the standard comparison machinery asserts output equality against the stored 3.x results. Configs that intentionally stay 3.x (birefringence) run unmigrated and are flagged via metrics['config_migrated'].
Parametrized end-to-end test: every 3.x configuration in docs/ and benchmarks/ is migrated and run on a small synthetic input; outputs must match the 3.x run array by array. A guard test fails if a new config directory appears without coverage. Shared path lists move to tests/config/repo_configs.py.
3.x configs are accepted with a deprecation warning for two releases (4.0 and 4.1), then rejected with a pointer to migrate-config.
In migrated-config mode the reconstruction runs from config_v4.yml, so the optimizer writes config_v4_optimized.yml; check_reference now reads either name. Caught by the migrated-mode regression run (case ops_pheno_center_opt, slurm 37801387).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the first five steps of the WaveOrder 4.0 design document on the
wo-4.0integration branch: the five-block configuration schema,migrate-config, version-aware loading,calibrate-tf, and a minimalsimulate. The design section 2 compatibility guarantee holds: every 3.x phase and fluorescence configuration in the repository and the published examples migrates mechanically and reproduces its 3.x reconstruction, bit for bit in the end-to-end tests and metric for metric in the regression suite. 3.x configurations keep running with a deprecation warning for two releases (4.0 and 4.1), then will be rejected with a pointer tomigrate-config; birefringence intentionally stays 3.x this release. Regression: 20/20 cases pass with 3.x configs (all 16 with a like-for-like baseline value-for-value equal to the 2026-10-28 pre-refactor run), and 20/20 pass again in migrated-config mode with every non-timing metric identical (302 metrics, slurm 37801387 and 37801460). Several design features parse against the schema but raiseNotImplementedErrornaming the gap (list at the end).Configuration (design sections 2 to 4)
waveorder/configpackage: pydantic schema with the five blocks (data,imaging_system,object,noise,solver) plusoutputsand a requiredversion; a configuration withoutversionis 3.x.waveorder migrate-config(aliasmc): deterministic, lossless 3.x to 4.0 migration for phase and fluorescence configurations.version: 4files run through the schema; 3.x files run exactly as before and print their 4.0 equivalent once with a deprecation warning stating the sunset policy above.Backwards-compatibility evidence (design section 2)
v4_to_legacy(migrate(cfg))reproduces the directly parsed 3.x settings field for field over every configuration in the repository; a guard test fails if a configuration directory appears without coverage.docs/examplesandbenchmarks/configs(including the optimization examples) is migrated and run end to end on synthetic input; the 4.0 output equals the 3.x output array by array, exactly.wo bm run --migrate-configsconverts each case's configuration withwo migrate-configand reconstructs from the 4.0 file, so the standard comparison machinery asserts output equality. Expanded 20-case suite (HPC, intel+GPU, slurm 37801387; one case rerun as 37801460 after a benchmark-harness filename fix for the optimizer's config_v4_optimized.yml): 20/20 pass, 302 non-timing metrics identical to the 3.x-config run (slurm 37771020), including the optimizer's recovered parameters. 17 cases ran from migrated configs; the birefringence and tile-stitch cases run unmigrated by design.Regression per-case verdict (3.x configs, slurm 37771020; "==" means the metrics equal the 2026-10-28 pre-refactor baseline exactly)
Calibration (design sections 3 and 6)
waveorder/calibratepackage: named Zernike terms (Noll indexing, orthonormal normalization, coefficients in waves), differentiable fluorescence PSF/OTF synthesis, the exact Whittle/Gaussian evidence, closed-form GCV for the Wiener filter, and the calibration penalty from parameter tolerances.wo calibrate-tf: estimates freepupil_aberrationsterms and writes atf.zarrwithparameters,parameters_covariance,parameter_names,tile_grid, andprovenance.score: sharpnessfolds in the existing patchwise Zernike recovery (design cases 5 and 17);score: evidenceandscore: gcvrun grid/random/gradient search stages with Laplace covariance,sigma: estimate, andweight: auto.object: {prior: {beads: {diameter}}}): joint MAP point localization fit together with the free pupil terms; the known object breaks the global sign-flip ambiguity, so bead calibration recovers signed coefficients.tf.zarrpassed with-tmoves the prior center and its covariance replaces the configured tolerances;--initsets the start only.Simulation (design section 5, minimal)
wo simulate: fluorescence truth system,gaussian_spectrumobject source, gaussian (sigma|snr_db) or poisson (photons) noise; writesdata.zarr,object.zarr,tf.zarrin the standard formats; a numeric tolerance draws the truth from its prior under the configuration's seed.Units. Zernike coefficients in waveorder (configs,
tf.zarr,waveorder.calibrate) are in waves of RMS wavefront error (orthonormal Noll). Pipelines that exchange coefficients in radians of pupil phase must convert withc_radians = 2 * pi * c_waves; the docstrings carry a prominent warning.Testing. 110+ new unit tests: schema validation, migration round trips over every repo config, the migrate-and-run equivalence tests above, Zernike conventions, synthesis vs the shift-variant model, evidence/GCV recovery, bead localization, tf.zarr round trips, fresh-interpreter import order, and the migrated-mode benchmark runner. Full suite green: 849 passed, 1 skipped, 2 xfailed under xvfb; the 12 cuda-parameterized tests pass on a GPU node (44/44 in their files).
Independent verification (sprint verifier, worktrees and venvs separate from the author's): branch vs merge-base on the same node, same suite code: 19/19 comparable cases identical on every non-timing metric (112+ metrics per case, including all histogram bins; the 20th case cannot run on base because base predates the tile-stitch forwarding this PR adds to bench.py). Branch vs the 2026-10-28 pre-refactor save-all baseline: 20/20 identical. Cold-process bit-identity of 3.x vs migrated 4.0 configs reproduced independently (max abs diff exactly 0.0; migrate-config byte-deterministic).
whittle_evidencevalidated against a dense exact Gaussian NLL to 4e-15. tf.zarr on-disk layout checked against design section 3 on the evidence, sharpness, and beads paths. Bead and sequential-tcalibration recover known injected aberrations (signed error 0.004 waves) on data from an independent simulator. The verifier's ranked review findings are all fixed on this branch (loud failure for free parameters without asolver.calibrationblock, configurable bead-detection thresholds, design-4.5 validations, singular-tcovariance rejected, grid ranges centered on the moved prior, documented DC-term exclusion and profile-curvature covariance).Known deviation from the design text. The design lists
padding: {z: 5, yx: 0}among the "defaults reproduce 3.x behaviour" items, but the 3.xz_paddingdefault is 0. The schema keepsz: 0(behavior preserving); the design text should be corrected.Not implemented yet (parses against the schema, fails with a
NotImplementedErrornaming the gap):reconstructdoes not yet chaincalibrate-tf, andapply-inv-tfdoes not read the newtf.zarr; tiled + evidence/gcv (tiled runs with sharpness only); phase-contrast calibration; confocal pinhole execution; NA/tilt/z-offset on the evidence path (they run via the 3.x sharpness optimizer);score: sure;gradient.method: alternating|joint;average_near_best_H; multi-inputcalibrate-tf;tikhonov_spectral/denoiserpriors,fista,nonnegon the new paths;poisson_gaussianin scores;dark_frames; simulate extensions (phase, phantom/from_recon sources, tiled truth,-ttruth). Birefringence configurations intentionally stay 3.x this release.