Skip to content

distance_transform_edt not always working with cuCIM backend #7660

Closed

Description

@aaronkujawa

Describe the bug
The distance_transform_edt returns all zero in special cases when cuCIM is installed and used as a backend.
This happens when convert_to_cupy() decides that the data array needs to be copied here:

To Reproduce

!pip install torch
!pip install cucim
!pip install monai
!pip install numpy

from monai.transforms import distance_transform_edt
import torch
import numpy as np

# create a c-contiguous array
mask = np.zeros((1, 4, 4))

# make it f-contiguous to force convert_to_cupy to copy the data array later on
mask2 = np.asfortranarray(mask)

# convert to torch tensor to make distance_transform_edt use cuCIM backend
mask = torch.tensor(mask, device='cuda')
mask2 = torch.tensor(mask2, device='cuda')

# set half of the mask to foreground
mask[0, 0:2, :,] = 1
mask2[0, 0:2, :,] = 1

# confirm masks are the same
print(mask)
print(mask2)

# calculate distance transform
edt = distance_transform_edt(mask)
edt2 = distance_transform_edt(mask2)

# check if the results are the same
print(edt)
print(edt2)
tensor([[[1., 1., 1., 1.],
         [1., 1., 1., 1.],
         [0., 0., 0., 0.],
         [0., 0., 0., 0.]]], device='cuda:0', dtype=torch.float64)
tensor([[[1., 1., 1., 1.],
         [1., 1., 1., 1.],
         [0., 0., 0., 0.],
         [0., 0., 0., 0.]]], device='cuda:0', dtype=torch.float64)
tensor([[[2., 2., 2., 2.],
         [1., 1., 1., 1.],
         [0., 0., 0., 0.],
         [0., 0., 0., 0.]]], device='cuda:0')
tensor([[[0., 0., 0., 0.],
         [0., 0., 0., 0.],
         [0., 0., 0., 0.],
         [0., 0., 0., 0.]]], device='cuda:0')

Expected behavior
Results should be the same, but the second output is all zero.

Activity

  1. added a commit that references this issue on Apr 19, 2024
    a4989fb
  2. KumoLiu commented on Apr 19, 2024

    @KumoLiu
    Contributor

    Hi @aaronkujawa, thanks for the reporting. I have created a PR to fix it.
    Main reason is that indices_ may not always be a shallow copy for F-order input as it constantly converts to C-order.

    indices_ = convert_to_cupy(indices)

    data = cp.ascontiguousarray(data)

  3. aaronkujawa commented on Apr 22, 2024

    @aaronkujawa
    ContributorAuthor

    Hi @KumoLiu , thanks for the quick fix!

  4. added a commit that references this issue on Apr 23, 2024
    ac9b186
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions