Skip to content

fix: stop seeding the DSC accumulator with a zero - #99

Open
Hendrik-code wants to merge 1 commit into
mainfrom
hm/fix-monai-dsc-bias
Open

Hendrik-code wants to merge 1 commit into
mainfrom
hm/fix-monai-dsc-bias

Conversation

@Hendrik-code

Copy link
Copy Markdown
Collaborator

The defect

dsc_list = [0]
...
    if dsc > 0:
        dsc_list.append(dsc)
...
return loss.mean().item(), np.mean(dsc_list)

The accumulator starts with a zero that is not a score, so every reported DSC averages one in. Ten batches at 0.8 report 0.727. The bias shrinks as the epoch lengthens, which is exactly why it reads as "our DSC is oddly low" rather than as a bug.

val_dsc is what gates checkpoint saving (if val_dsc > val_dsc_best: torch.save(...)), so this sat directly on model selection.

The fix

Start empty. An epoch in which nothing scored above zero reports nan rather than 0.0, which would be indistinguishable from a real score of zero.

Tests

unit_tests/test_monai_dsc_bias.py drives the real train() with identical batches, so every per-batch DSC is the same known value and the epoch mean must equal it — and asserts explicitly that it is not the phantom-zero average. (monai/wandb are stubbed as in test_monai_epoch_metrics.py; train() needs neither.)

The empty-loader case is asserted on the helper rather than through train(), because an empty loader also trips the separate epoch-loss defect — that one is fixed in its own PR, and the test says so.

Verified failing before the fix and passing after. Full suite unchanged against baseline; ruff check . clean.

🤖 Generated with Claude Code

Both epoch loops started from `dsc_list = [0]` and then appended only scores
above zero, so every reported DSC averaged in one phantom zero: ten batches at
0.8 report 0.727. The bias shrinks as the epoch lengthens, which is why it reads
as "our DSC is oddly low" rather than as a bug.

`val_dsc` is what gates checkpoint saving (`if val_dsc > val_dsc_best`), so the
bias sat directly on model selection.

An epoch in which nothing scored above zero now reports nan rather than 0.0,
which would be indistinguishable from a real score of zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant