You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
GlobalMutualInformationLoss returned -inf during training in my multi-model MRI image registration with VoxelMorph, which led to the failure of training. GlobalMutualInformationLoss was constructed with default parameters. What caused the issue and What can be done to deal with it?
After debugging, I found that it was caused by matrix multiplication with float16 precision in GlobalMutualInformationLoss when using Automatic mixed precision (AMP). without AMP, the loss values seemed normal.
Activity
kvttt commentedon Apr 11, 2024
I am not sure if this issue is caused by
VoxelMorphbut if that is the case, I am happy to help :)jslalu commentedon Apr 12, 2024
After debugging, I found that it was caused by matrix multiplication with float16 precision in GlobalMutualInformationLoss when using Automatic mixed precision (AMP). without AMP, the loss values seemed normal.
