Training issue

[yujingxin]

I tried training from scratch using the default configuration but without text prompt. How significant is the impact of not using text annotations? Training without prompts results in outputs that are somewhat similar but exhibit reduced sharpness and clarity.

[Feynman1999]

I tried training from scratch using the default configuration but without text prompt. How significant is the impact of not using text annotations? Training without prompts results in outputs that are somewhat similar but exhibit reduced sharpness and clarity.

Could you provide more details about the training setup?

1. Are you currently using empty prompts for both the generator and the discriminator?

2. Is the training dataset the same as the one used in the paper?

3. After training, did you run inference and compare the results with the official checkpoint to reach this conclusion?

[yujingxin]

thank u for replying.

1. yes. use empty prompts for both the generator and the discriminator
2. yes. same as the one used in the paper
3. yes. also test with the official checkpoint

[yujingxin]

@Feynman1999 Additionally, I’d like to ask another question: I fine-tuned the model with the official checkpoint, performed just one gradient update, saved the model immediately, and then ran an evaluation. I noticed that the deblurring performance dropped significantly—and was quite similar to the results I got when training from scratch. Do you have any suggestions regarding this issue?

[Feynman1999]

1. Based on the information you provided, my understanding is that the performance difference is most likely caused by training with empty prompts. In the paper, we indeed did not include an ablation study analyzing the effect of setting all prompts to empty during training.

2. In the future, we will organize and release our training dataset (512×512 crops from LSDir/FFHQ, along with the corresponding prompts) to help the community reproduce the training process.

3. Regarding your comment about loading the model and evaluating the result after only one iteration, the outcome is mainly affected by two factors:
   1️⃣ Since this is adversarial training, the discriminator is in a poor initial state, which negatively impacts the generation quality.
   2️⃣ If the corresponding optimizer states are not loaded, the performance will also be affected; typically, it requires dozens to hundreds of iterations before the results start to improve.