About FLOPs and running time

[ChimpOnCloud]

Great work,
I tested your code on DiT-XL-2-256x256,
with command

python sample.py --image-size 256 --num-sampling-steps 250 --cache-type attention --fresh-threshold 4 --fresh-ratio 0.07 --ratio-scheduler ToCa-ddpm250  --force-fres
h global --soft-fresh-weight 0.25 --seed 1234 --test-FLOPs

It shows 43.420574210048 TFLOPs Total Sampling took 9.7924873046875 seconds

with command

python sample.py --image-size 256 --num-sampling-steps 250 --cache-type attention --fresh-threshold 1 --fresh-ratio 0.07 --ratio-scheduler ToCa-ddpm250  --force-fres
h global --soft-fresh-weight 0.25 --seed 1234 --test-FLOPs

It shows 103.549585270784 TFLOPs Total Sampling took 10.506012695312501 seconds

So the FLOPs were cut to nearly half but running time seems not changed apparently. What is the probable reason of this phenomenon?

I ran the above code on a single NVIDIA 3090 RTX.

[Shenyi-Z]

Hello, thank you for your interest in our work! It is normal for the latency acceleration to be insignificant when using sample.py because, in single inference, warm-up factors significantly affect the image generation speed. We recommend testing with multiple images or using the single-GPU mode of sample_ddp.py for large-batch inference, where the latency acceleration will be more noticeable.

[haoweiz23]

Hi, @Shenyi-Z

I tested the inference time using sample_ddp.py in single-GPU mode with a batch size of 16 on 320 samples and obtained the following average inference times:

• PixArt-alpha: 0.4624 seconds for 320 samples
• With ToCa (fresh_ratio = 0.3): 0.3011 seconds for 320 samples

This results in a 1.53× acceleration, which is lower than the 1.75× acceleration (0.390s vs. 0.682s) reported in Table 1 of your paper.

Could you provide more details or share the evaluation code used to measure inference speed?

[haoweiz23]

In addition, I tested the baseline using --fresh-threshold 1 --fresh-ratio 1.0. Would this differ from the original baseline?

[Shenyi-Z]

Hi, @Shenyi-Z

I tested the inference time using sample_ddp.py in single-GPU mode with a batch size of 16 on 320 samples and obtained the following average inference times:

• PixArt-alpha: 0.4624 seconds for 320 samples
• With ToCa (fresh_ratio = 0.3): 0.3011 seconds for 320 samples

This results in a 1.53× acceleration, which is lower than the 1.75× acceleration (0.390s vs. 0.682s) reported in Table 1 of your paper.

Could you provide more details or share the evaluation code used to measure inference speed?

Hello! The inference for 320 samples on PixArt should be completed quickly. You may want to check whether the device is sufficiently warmed up. In our paper, latency was tested under the following conditions: with a batch size of 100, we averaged the inference time after processing approximately 30k images (as in our provided prompt set). This ensures that the device's computing power is fully utilized and that the warm-up time can be ignored. Additionally, latency itself is highly dependent on the device and runtime environment. Therefore, the acceleration ratio calculated using FLOPs in our paper may be more reliable and generalizable.

[Shenyi-Z]

In addition, I tested the baseline using --fresh-threshold 1 --fresh-ratio 1.0. Would this differ from the original baseline?

There might be a minor issue here. Our recommended approach is to set fresh-ratio to 0.0. However, since fresh_threshold has already been set to 1, considering the scheduler in this file, this should not cause any additional steps to be skipped. Therefore, it is valid to test it in this way.