Skip to content

Feature Request: Evaluation of Semantic Segmentation Metrics on a per-component basis #8733

Closed

Description

@alexanderjaus

Is your feature request related to a problem? Please describe.

Current MONAI segmentation metrics aggregate scores globally. In multi-instance medical segmentation tasks this can mask clinically relevant failure modes, e.g. missing small lesions while achieving high Dice scores due to dominant large structures. There is no native support for connected component aware evaluation except Panoptic Quality that treats each ground truth object as an individual evaluation unit.

Describe the solution you'd like

Add connected component aware variants of existing MONAI semantic segmentation metrics. The proposed CC (Connected-Component-wise) Metrics operate as thin wrappers around existing MONAI metrics and:

  • Decompose the ground truth foreground into connected components
  • Assign each voxel to its nearest ground truth component via a Voronoi partition based on a single Euclidean distance/feature transform.
  • Evaluate standard MONAI metrics inside each component specific region of interest.
  • Supports aggregation modes (patient: mean over components per case & overall: flat aggregation over all components)
  • Integrate seamlessly with existing MONAI metrics such as DiceMetric, HausdorffDistanceMetric, SurfaceDistanceMetric, and SurfaceDiceMetric.

The implementation may be binary segmentation only first but coule be extended to multi-class approaches.

Describe alternatives you've considered

  • Instance segmentation metrics. These require different model outputs and are not applicable to semantic segmentation pipelines.
  • Panoptic Quality offers a part of the solution but relies on a fixed and threshold dependant overlap-based matching and only consideres IoU as a metric.

Additional context

Reference implementation exists as a standalone library CC Metrics (https://github.com/alexanderjaus/CC-Metrics), published with peer reviewed validation.

The approach is described in “Every Component Counts: Rethinking the Measure of Success for Medical Semantic Segmentation in Multi Instance Segmentation Tasks”, AAAI 2025.

Activity

  1. VijayVignesh1 commented on Mar 11, 2026

    @VijayVignesh1
    Contributor

    Hi @alexanderjaus,
    I can start with integrating the per component parameter for the DiceScore metric, if that's okay.

  2. added a commit that references this issue on May 1, 2026
    65beb58
  3. VijayVignesh1 commented on May 7, 2026

    @VijayVignesh1
    Contributor

    @ericspod There are other metrics pending in the issues (DiceMetric, HausdorffDistanceMetric, SurfaceDistanceMetric, and SurfaceDiceMetric). I've raised a PR for HausdorffDistanceMetric and I'll do one for SurfaceDistanceMetric as well as SurfaceDiceMetric.

  4. added a commit that references this issue on May 25, 2026
    1e165ed
  5. added a commit that references this issue on Oct 19, 2026
    33bcc40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions