Repository navigation
feat: noise relative to the sample's own spread with RandomGaussianNoiseGPU(relative=True) - #147
Hendrik-code wants to merge 1 commit into
Conversation
…iseGPU(relative=True) The noise std is drawn from U(0, std) in absolute intensity units, so one config is a different augmentation on every intensity scale. With the paper config's std 1.0 a z-scored MR patch (std ~0.9) gets noise up to about its own spread, a CT patch under nnU-Net's CT normalisation (std ~0.17) up to six times its spread, and a [0, 1] SynthSeg synthesis (std ~0.18) likewise. `relative=True` reads `std` as a fraction of each sample's std in that channel, which makes the transform scale-equivariant. The scale is applied after both random draws, so the noise field is the absolute one rescaled per sample. `mean` stays absolute. Default False; the absolute path is arithmetically unchanged (the per-sample and batch stds now broadcast through the same reshape, which multiplies by the same value). Also corrects the docstring, which gave the default std as 0.1; it is 1.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Results: Dataset803 and Dataset804, fold 0Relative noise with a cap of 0.2 sample stds is equivalent to the paper config on Dataset803. On Dataset804 out-of-domain it is slightly worse, mostly in the Dixon water cohort. The arm changes two things at once: the noise becomes scale-free, and it becomes much weaker on CT. So the result argues against weakening the noise, not against making it relative. Setup.
Relative 0.2 minus paper (replicate), paired mean and 95% CI:
Three of the four cells are equivalent (TOST p < 0.001). On Dataset804 out-of-domain the point estimate is inside the margin but the interval is not bounded by it (TOST p = 0.44). Against the published run, the four deltas are +0.001, −0.006, +0.000 and −0.007, all equivalent. Cohort means: 803 in / out 0.902 / 0.841 (paper replicate 0.901 / 0.844). 804 in / out 0.903 / 0.766 (paper replicate 0.903 / 0.781, so −0.016 out-of-domain). Where it moves, per cohort against the replicate:
The loss lands on the other modality in both datasets: CT for the MRI-trained model, MRI for the CT-trained one. On CT patches (std about 0.17), the paper's absolute cap of 1.0 is about 6 sample stds; this arm's cap is 0.2 stds, about 30 times weaker. On z-scored MRI (std about 0.9) it is about 5 times weaker. This fits the heavy CT noise acting as a useful appearance augmentation rather than a defect. Caveat: GPU model. This arm's Dataset804 run trained on an L40S; every reference trained on an A40. On Dataset804 Dixon water, the three arms of this round that trained on the L40S (#147, #149, #150) scored 0.724, 0.698 and 0.539. That is below all 23 other full-capacity arms on 804 (0.756 to 0.778) and all four runs of the paper config (0.767 to 0.771). An earlier paper-family run on the same L40S scored 0.759, with no scan more than 0.05 below the paper config, so the card alone does not produce the drop. With one run per cell, an interaction between card and arm is not excluded; an A40 rerun on 804 would settle it. In training. About 36 s/epoch on the L40S, no non-finite losses in either run. At p = 0.1 the noise fired in 1 of 20 pre-flight batches, so the dose check forced p = 1 on both sides. Bottom line: the option defaults to off and is safe to merge. The result does not justify replacing the paper's absolute noise. To test scale-free noise on its own, the relative cap would need to match the old dose on CT, about 1 to 6 sample stds. |
What
RandomGaussianNoiseGPUtakesrelative(defaultFalse). When set,stdis read as a fraction of each sample's own std in that channel, not in absolute intensity units:Why
The std is drawn from
U(0, std)in absolute units, so one config is a different augmentation on every intensity scale. Measured on 128³ training patches, image std is about 0.9 on z-scored Dixon MR, 0.17 on CT under nnU-Net's CT normalisation, and 0.16-0.20 on a[0, 1]SynthSeg synthesis. The paper config'sstd: 1.0therefore means:For scale, nnU-Net's default noise std is at most 0.1 on z-scored data (from memory of batchgenerators'
noise_variance=(0, 0.1), which it uses as the std).Behaviour
meanstays absolute.Tests
unit_tests/test_gaussian_noise_relative.py:Full suite:
477 passed, 13 skipped.mypy smauglab/clean, pre-commit passes.🤖 Generated with Claude Code