Skip to content

Please add another interpolation mode for NiftiSaver to avoid ziazag pattern caused by 'nearest' #3333

Closed

Description

@Jingnan-Jia

Is your feature request related to a problem? Please describe.
In the current NiftiSaver, there are only two interpolation modes: "nereast" and "bilinear". It is not enough. Detailed explanication as follows.

If I have an lung CT image with shape (1024, 512, 512). To do lung or lobe segmentation, I need to down sample the original CT image to a smaller size (256, 256, 256). So the resample factors are (0.25, 0.5, 0.5). After I finish the model training, the predicted lung or lobe segmentation needs to be up-sampled to the original shape during inference. If I use "bilinear" mode for NiftiSaver, some zigzag pattern would appear along the edge of lung or lobe.

Describe the solution you'd like
Using current MONAI, I came up with several possible solutions to avoid the zigzag pattern in NiftiSaver :

  1. Use "bilinear" mode in NiftiSaver at first, and then use a threshold of 0.5 to binarize the results. However this method only applies to binary segmentation tasks, like lung segmentation. In multi-label segmentation, there is not a proper threshold for the whole image.
  2. I can use "nearest" mode at first to get the output with zigzag pattern. Then use some smooth algorithms to remove the zigzag pattern.

I do not have a satisfied solution in current MONAI. I hope NiftiSaver can have a mode "bilinear2nearest". In this mode, the NiftiSaver will:

# In a n-label image segmentation
1. apply `argmax` to prediction, get the discreted masks.
2. apply `on_hot_encode` to the discreted masks, to get `n` channels of binary masks.
3. apply `bilinear` interpolation mode to eac channel of binary mask, get `n` channels of masks of original size.
4. apply `threshold=0.5` to all channels, get `n` channels of binary masks.
5. apply `on_hot_decode` to get the final n-label mask.

I have implementated this workflow in my own code, and it works well. I hope MONAI can also add this feature to NiftiSaver.

Note: The zigzag pattern matters especially when the interpolation factor is far from 1.

Note: this initial idea was implementated by myself these days and this idea can not remove zigzag pattern. So I followed [#3178 ], changed his Gaussian filter to Mean filter, and then got satisfied results. More details were shown bellow.

Activity

  1. wyli commented on Nov 14, 2021

    @wyli
    Contributor

    related to #3178

  2. Nic-Ma commented on Nov 14, 2021

    @Nic-Ma
    Contributor

    Hi @Jingnan-Jia ,

    Thanks for your interesting experiments and detailed feedback.
    If I understand correctly, is it possible to execute your operations with several transforms with interpolation modes?
    For example: Resized, AsDiscreted, SaveImaged, etc.

    Thanks.

  3. Jingnan-Jia commented on Nov 19, 2021

    @Jingnan-Jia
    Author

    Hi @wyli and @Nic-Ma , Thanks for your reply.

    I have tried some experiments and found that my expectated feature is almost the same with [#3178 ].

    The only difference is that the implementation of [#3178 ] is using Gaussian filger, while my implementation is using Mean filter. And my threshold is also 0.4. (By the way, my task is 3D lung lobe segmentation.)

    I have tried his Gassian filter but it did not work. Maybe because I did not set the correct sigma value.

  4. Jingnan-Jia commented on Nov 19, 2021

    @Jingnan-Jia
    Author

    微信图片_20211119114856

    To have a better view, please click the figure and see the original size with high resolution.

  5. Jingnan-Jia commented on Nov 19, 2021

    @Jingnan-Jia
    Author

    The left figure is the original mask. The middle figure is the results after down-sampling and up-sampling. Obvious ziazzag pattern appeared at the edge. The right figgure is the result after I apply my smooth algorithm to the middle figure. We can see the significant improvement.

    The dice between figure 1 and figure 2: 0.982
    The dice between figure 1 and figure 3: 0.989

  6. Jingnan-Jia commented on Nov 19, 2021

    @Jingnan-Jia
    Author

    However, now I am not sure if we should add this feature to MONAI. Because it is not easy to select a good smooth hyper parameter or threshold for different masks with different resolutions.

  7. wyli commented on Oct 5, 2022

    @wyli
    Contributor

    closing this in favour of #3333 and #5264

    NiftiSaver is also deprecated, the current preferred way is to have a postprocessing transform sequence:

    https://github.com/Project-MONAI/tutorials/blob/8abcbc62c704af0bb22bedba617ae914c7802b97/modules/engines/unet_evaluation_dict.py#L89-L96

    val_post_transforms = Compose(
        [
            EnsureTyped(keys="pred"),
            Activationsd(keys="pred", sigmoid=True),
            AsDiscreted(keys="pred", threshold=0.5),
            KeepLargestConnectedComponentd(keys="pred", applied_labels=[1]),
            SaveImaged(keys="pred", resample=False, output_dir="./runs/")
        ]
    )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions