Skip to content

GlobalMutualInformationLoss goes to -inf #7622

Open

Description

@jslalu

GlobalMutualInformationLoss returned -inf during training in my multi-model MRI image registration with VoxelMorph, which led to the failure of training. GlobalMutualInformationLoss was constructed with default parameters. What caused the issue and What can be done to deal with it?

Activity

  1. kvttt commented on Apr 11, 2024

    @kvttt
    Contributor

    I am not sure if this issue is caused by VoxelMorph but if that is the case, I am happy to help :)

  2. jslalu commented on Apr 12, 2024

    @jslalu
    Author

    After debugging, I found that it was caused by matrix multiplication with float16 precision in GlobalMutualInformationLoss when using Automatic mixed precision (AMP). without AMP, the loss values seemed normal.
    1712883159954

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions