About the inconsistency in GPU memory usage.

[Starry-16]

Hello, your work is excellent, but I have a question that I'm not sure if you can help me with. I noticed that using ToCa on the DiT model causes additional memory overhead. I understand that this method introduces extra space to store caches, but when using Flux, the memory usage seems to decrease. Why is that?

[Shenyi-Z]

Sounds interesting. I guess it’s probably either because the VRAM is maxing out (triggering PyTorch's auto-management), or you didn't empty the cache/garbage before measuring.