Repository navigation
maisi unconditioned inference return noise #2017
Description
Hallo,
I tried longer to make the diffusion unet (rectified flow version) work but without success. the VAE performes correct and encoding and reconstruction look fine. I take all the configs from the repo so configuration as well as weights are equivalently loaded as in https://github.com/Project-MONAI/tutorials/blob/main/generation/maisi/maisi_inference_tutorial.ipynb.
Using the diffusion unet with the existing code of diffusion_model_infer.py with
from scripts.diff_model_infer import run_inference
data = run_inference(
args,
device,
autoencoder,
diffusion_unet,
1.4,#scale_factor,
None,#top_region_index_tensor,
None, #bottom_region_index_tensor,
torch.tensor([[1.5,1.5,4.0]]).to('cuda'),#spacing_tensor,
torch.tensor([1]).to('cuda'),#modality_tensor,
(96,96,96),#output_size,
4,
logger,
)
I saw that there was an issue with the architecture which was different during training and the implementation but I'm not sure if that is related.
Project-MONAI/MONAI#7991.
Some observations I made is when I would like to just reconstruction an image and basically make one step, the recon get's extremly noise.
For any suggestion what could be the potential issue, I'm thankful.
Best,
Lukas
Activity
Can-Zhao commentedon Aug 14, 2025
scale_factor is loaded from checkpoint. output_size can not be [96,96,96]. It needs to be divisible by 128
LFetty commentedon Aug 18, 2025
Thanks for the quick answer. I indeed tested more and came also to the conclusion that higher output resolution helps a lot! Sorry I did overread that information in the paper. However, using the smallest possible output size of 128x128x128 have still very bad results. I guess the reason is that there are only very few training cases with that resolution level.
In the paper I did not find info about what the training resolution of the diffusion unet was. As the resolution would be very difference for all volumes was the latent representation cropped to a specific size?
Can-Zhao commentedon Jan 30, 2026
128x128x128 performs badly for ddpm, but it should be fine for rflow