flux-toca is missing parameters cache_dic and current

[anfengmin100]

When I run flux-toca, it shows that there are no cache_dic and current related parameters.

It looks like you are not passing these two parameters

What parameters should be passed to these two parameters?

[Shenyi-Z]

Hello! Thank you for your interest in our work! I can roughly understand the possible issue here. Could you provide more detailed information about the error for further troubleshooting?

[anfengmin100]

Hello! When I use python -m flux --name flux-dev --height 1024 --width 1024 --prompt "a girl" to run,
It shows that "TypeError: 'NoneType' object is not subscriptable" in ToCa/flux-ToCa/src/flux/model.py/line 121,
The code is "cal_type(cache_dic=cache_dic, current=current)".
When I went to find out why cache_dic is None,I found that you defined it this way，
in ToCa/flux-ToCa/src/flux/model.py/line 84
def forward(
self,
img: Tensor,
img_ids: Tensor,
txt: Tensor,
txt_ids: Tensor,
timesteps: Tensor,
y: Tensor,
guidance: Tensor | None = None,
*args,
**kwargs,
) -> Tensor:
if img.ndim != 3 or txt.ndim != 3:
raise ValueError("Input img and txt tensors must have 3 dimensions.")

    print(kwargs)
    cache_dic = kwargs.get('cache_dic', None)
    current = kwargs.get('current', None)

You passed these two parameters via **kwargs，But when you actually call the method, you don't pass the **kwargs parameter，
In ToCa/flux-ToCa/src/sampling.py/line 269 you do
pred = model(
img=torch.cat((img, img_cond), dim=-1) if img_cond is not None else img,
#img_ids=img_ids[1] if small else img_ids[0],
img_ids=img_ids,
txt=txt,
txt_ids=txt_ids,
y=vec,
timesteps=t_vec,
guidance=guidance_vec,
)
So I don't know what the values ​​of cache_dic and current are, they are both None.

[Shenyi-Z]

Oh, I understand now. Thank you for your clear explanation! I haven't adjusted the loop and single calling methods here yet. You can temporarily consider using the method in src/sample.py for inference. I will resolve this issue soon!

[Shenyi-Z]

Hello, please check the latest version. This issue should be resolved now.

[anfengmin100]

Thank you for your repair！But I have discovered a new problem，in ToCa/flux-ToCa/src/flux/ideas/cache_denoise.py line 3 from .split_utils import split, merge. It shows that No module named 'flux.ideas.split_utils', I can't find split_utils.py

[algorithmconquer]

Thank you for your repair！But I have discovered a new problem，in ToCa/flux-ToCa/src/flux/ideas/cache_denoise.py line 3 from .split_utils import split, merge. It shows that No module named 'flux.ideas.split_utils', I can't find split_utils.py

I met the same problem!!!

[algorithmconquer]

Thank you for your repair！But I have discovered a new problem，in ToCa/flux-ToCa/src/flux/ideas/cache_denoise.py line 3 from .split_utils import split, merge. It shows that No module named 'flux.ideas.split_utils', I can't find split_utils.py

You can remove the line "from .split_utils import split, merge"!

[Shenyi-Z]

Hello! You can directly delete this line of code. I have already made the adjustment in the latest version.

[anfengmin100]

Thank you for your repair！But I have discovered a new problem，in ToCa/flux-ToCa/src/flux/ideas/cache_denoise.py line 3 from .split_utils import split, merge. It shows that No module named 'flux.ideas.split_utils', I can't find split_utils.py

You can remove the line "from .split_utils import split, merge"!

Thanks! It works!!!

I compared the effects of Flux and Flux-ToCa, The quality of the generated image is seriously degraded.My parameters are as follows
model: Flux-dev
Prompt: Travel photography,romance, wide-angle, The image captures a cityscape at dusk, featuring a train passing through the urban area. The train is illuminated, adding a warm glow to the scene. In the background, there are several buildings, including a prominent hotel with a sign that reads "INN." The city appears to be a mix of modern and older structures, creating a unique urban environment.
WidthHeight: 8641536
Steps: 50
Guidance: 3.5
Seed: 1
ToCa has an advantage in speed, 21.4s (Flux-ToCa) vs 34.6s (Flux), but it takes up more video memory, 43962M (Flux-ToCa) vs 53982M (Flux), and the image quality and details are greatly reduced. The building crashed after using the ToCa method in the picture.
Is this decline in quality normal?

[Shenyi-Z]

Emmm, the time and performance are both unexpected, quite strange. Please wait for two days, and I will resolve this issue as soon as possible after the ICCV submission deadline.

[Shenyi-Z]

Thank you for your repair！But I have discovered a new problem，in ToCa/flux-ToCa/src/flux/ideas/cache_denoise.py line 3 from .split_utils import split, merge. It shows that No module named 'flux.ideas.split_utils', I can't find split_utils.py

You can remove the line "from .split_utils import split, merge"!

Thanks! It works!!!

I compared the effects of Flux and Flux-ToCa, The quality of the generated image is seriously degraded.My parameters are as follows model: Flux-dev Prompt: Travel photography,romance, wide-angle, The image captures a cityscape at dusk, featuring a train passing through the urban area. The train is illuminated, adding a warm glow to the scene. In the background, there are several buildings, including a prominent hotel with a sign that reads "INN." The city appears to be a mix of modern and older structures, creating a unique urban environment. Width_Height: 864_1536 Steps: 50 Guidance: 3.5 Seed: 1 ToCa has an advantage in speed, 21.4s (Flux-ToCa) vs 34.6s (Flux), but it takes up more video memory, 43962M (Flux-ToCa) vs 53982M (Flux), and the image quality and details are greatly reduced. The building crashed after using the ToCa method in the picture. Is this decline in quality normal?

I have investigated the issue, and the timing discrepancy is reasonable. The reason why the timing does not match our project is that ToCa does not yet support FlashAttention. We will address this issue in the upcoming release.

As for the quality issue, it is quite strange. We ran the prompt you provided locally and obtained the results shown in the image. On the left is the original flux, and on the right is flux-ToCa. It appears that there is no significant quality loss.

[Shenyi-Z]

Hello!

I have rechecked our project, and simply cloning the repository and running it as expected should yield good performance. Have you modified any settings?

We recommend using the following command for execution:

python src/sample.py \
  --prompt_file /path/to/your/prompts.txt \
  --width 864 \
  --height 1536 \
  --model_name flux-dev \
  --add_sampling_metadata \
  --output_dir /path/to/save/image \
  --num_steps 50

Please try this and let us know if you encounter any issues.

[anfengmin100]

Hello!

I have rechecked our project, and simply cloning the repository and running it as expected should yield good performance. Have you modified any settings?

We recommend using the following command for execution:

python src/sample.py
--prompt_file /path/to/your/prompts.txt
--width 864
--height 1536
--model_name flux-dev
--add_sampling_metadata
--output_dir /path/to/save/image
--num_steps 50
Please try this and let us know if you encounter any issues.

Thanks for your reply!

According to the image you generated, when the image is enlarged, it can be seen that the window of the building in the upper left corner of the image generated by Toca flux is already broken, while the one generated by flux is normal. In addition, there is a sense of fragmentation in the streetlights, pedestrians on the road, and the ground. This situation is more pronounced in portrait images and other high detail scenes, and I can provide more comparison images.


In this case, Toca flux showed significant reductions in the character's headwear, clothing material, clothing patterns, facial details, and hair details. The parameters for this set of cases are as follows:
prompt: ba_wang_bie_ji, Farewell My Concubine of style1, ultra HD 8K, silk fabric texture visible, photorealistic, phase detection autofocus bokeh, The image depicts a person dressed in traditional Chinese opera attire. The costume is vibrant and elaborate, featuring a red robe adorned with intricate embroidery and patterns in various colors such as gold, blue, and green. The person is wearing a large, ornate headdress that is decorated with numerous small white balls and other decorative elements, including a butterfly and floral motifs. The headdress is quite heavy and elaborate, indicating the importance of the character being portrayed. The person is also wearing a mask, which is a common feature in Chinese opera to conceal the identity of the actor and emphasize the character's expression and emotions. The mask is painted with detailed facial features and is complemented by the costume. The person is holding a fan, which is another traditional accessory in Chinese opera. The overall appearance is very regal and theatrical, reflecting the grandeur and artistry of Chinese opera.
seed:1
steps:50
width&height:1360*768
In addition, using default prompt words for flux can also have the same problem.

You can see that the effects on trees, FLUX text, road surfaces, and fallen leaves are all very blurry, with a strong sense of fragmentation.
What is the reason for the significant decrease in generated image quality？