Regarding Reproducing Table 1 (N=3, R=60) Results from Your Paper (Pixart-alpha, MS-COC02017 FID-30K)

[applezoos]

Thank you for sharing your excellent research and providing the codebase.
I am currently attempting to reproduce the results from your paper, but I have encountered some difficulties.

Specifically, I am trying to reproduce the MS-COCO2017 FID-30k result of 28.02 reported in Table 1 for N=3 and R=60.
I have tested various options using the COCO_caption_prompts_30k.txt file you shared, but the resulting FID scores are not as good as expected.

Could you kindly share the exact command or options used to run the PixArt-alpha-ToCa/scripts/inference_ddp.py script to achieve this result?

I appreciate your time and assistance, and I look forward to your guidance.

[Shenyi-Z]

Thank you for your interest! Following the instructions in our GitHub project, setting the fresh_ratio to 0.40 should help you reproduce the N=3, R=60 result you are looking for.

Regarding the FID-30k result on MS-COCO mentioned in the paper, based on our experience, it is highly dependent on which 30k images are selected for comparison. We recommend using the compressed MS-COCO .npz file we provided earlier in a previous response. The link should still be active: https://drive.google.com/file/d/1vUdoSgdIvtXo1cAS_aOFCJ1-XC_i1KEQ/view?usp=sharing, to achieve as consistent a result as possible.

Additionally, please note that using FID and CLIP for evaluation on PixArt-alpha might not be entirely accurate. We are currently exploring whether there are more suitable metrics for evaluation.

[applezoos]

Thank you for your guidance! Following your advice, I was able to achieve the FID performance.