This repository was archived by the owner on Feb 7, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Feb 7, 2025. It is now read-only.
Class conditioned diffusion #486
Description
Hi!
Thanks for the great work here.
When working with class-conditioned diffusion, the available inferers don't offer an input of class_label, only a concat or crossatn based context. However the diffusion model has all the infrastructure needed and seems to handle class conditioning well.
I am wondering if this is intentional and I am misusing the class label or if there is a plan to add class_label inputs to the inferer classes.
Seems like a very minimal amount of code change if I understood correctly.
But I don't understand why I should use the context input instead of the class_label which seems more relevant.. did I misunderstand something?
Activity
ericspod commentedon May 3, 2024
@marksgraham or @virginiafdez may have further insight here, but it's possibly an oversight in the DiffusionInferer class we can rectify with the integration into core.
marksgraham commentedon May 3, 2024
Yeah I think we should be supporting input of a
class_labelin infererers. I suspect the reason we don't have it is that we found conditioning tends to work best through context, but we should support it and let users decide for themselves. I'll try to add it in on the refactor.@talg2324 we're moving MONAI Generative into MONAI core you'll have to wait for that port to be complete and the new feature will be available there
Ahmad-Omar-Ahsan commentedon May 9, 2024
I have a question regarding the scheduler used for class conditional sampling.
So I noticed in the tutorial that DDPM was used, can we use DDIM instead to generate the samples? If I use the same noise vector and labels
[0,1,2,3]for 4 disease conditions, I notice that the DDPM scheduler can generate 4 distinct images. Yet with DDIM it generates the same image. Can't we use DDIM to emulate the scenario that the images are from the same subject but with different disease conditions?marksgraham commentedon May 10, 2024
You can swap out the DDPM for DDIM scheduler during sampling. Sometimes, a model producing good samples with DDPM sampling will not produce good samples with DDIM sampling, this might indicate you need to train the model more in my experience
OdedRotem314 commentedon Oct 13, 2024
Class_label conditioning is not really implemented in full as far as I understand in 2d_ddpm_classifier_free_guidance_tutorial
virginiafdez commentedon Oct 14, 2024
Hello,
I'm sorry to hear that the conditioning is not working well.
The code for MONAI Generative Models has now been moved and re-implemented as part of MONAI core. If the issue persists with the MONAI Core code, could you please raise an issue there and we will have a look at it?
Thanks a lot!