How much VRM is needed to run flux-Toca？

[ziyaxuanyi]

When I run flux-Toca on RTX3090-24G，it will lead to out of memory.

Is it because the cache to keep all blocks of all tokens requires too much memory that causes the memory to run out？

[Shenyi-Z]

Hello! Thank you for your interest in our work!
Based on our estimation, the RTX 3090 with 24GB of VRAM is likely insufficient for running inference on the FLUX.1 model and may require quantization or other optimizations. Additionally, ToCa's cache does increase storage requirements. We recommend using GPUs with at least 55GB of VRAM (such as A100/800, H100/800) for this part of the experiment.