Unable to run `cli_control.py` for ClusCa, SpeCa, and sampling for TaylorSeers

[AniAggarwal]

Thanks for creating this combined repo!

It appears that the cache_cutfresh_by_cluster method is not defined anywhere, resulting in errors when attempting to use the ClusCa code.

In SpeCa, there is a circular import issue where layers.py and model.py both attempt to import each other, and cli_control.py also does not run.

When running diffusers_taylorseer_flux.py, with the command:
python diffusers_taylorseer_flux.py --prompt "A sunset over a desert."
I encounter the following error:

Traceback (most recent call last):
  File "Cache4Diffusion/flux/taylorseer_flux/diffusers_taylorseer_flux.py", line 136, in <module>
    main()
  File "Cache4Diffusion/flux/taylorseer_flux/diffusers_taylorseer_flux.py", line 110, in main
    image = pipeline(
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py", line 696, in __call__
    noise_pred = self.transformer(
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "Cache4Diffusion/flux/taylorseer_flux/forwards/flux_forward.py", line 110, in taylorseer_flux_forward
    image_rotary_emb = self.pos_embed(ids)
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 65, in forward
    [rope(ids[..., i], self.axes_dim[i], self.theta) for i in range(n_axes)],
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 65, in <listcomp>
    [rope(ids[..., i], self.axes_dim[i], self.theta) for i in range(n_axes)],
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 44, in rope
    batch_size, seq_length = pos.shape
ValueError: not enough values to unpack (expected 2, got 1)

I am on diffusers v0.30.3. I did not see a requirements.txt file, is there a particular version I should use?

Thank you for the help!

[Shenyi-Z]

Thanks for your share! We will fix it as soon as possible~

[Shenyi-Z]

Regarding the issue with the diffusers version, we will continuously maintain the compatibility of our code with the latest version of diffusers. In the future, we will also attempt to adapt to excellent open-source projects, such as ComfyUI and XDiT. Currently, using diffusers version v0.35.1 should work.

[AniAggarwal]

Got it, thank you! I am able to now run taylorseer on v0.35.1.
How does one control the N and O hyper parameters from the paper the paper?

[Shenyi-Z]

Got it, thank you! I am able to now run taylorseer on v0.35.1. How does one control the N and O hyper parameters from the paper the paper?

Always in the cache_init.py, and we will convert it into a command line parameter in a future version.

[AniAggarwal]

Thank you for the quick response.
To confirm, if I wanted to modify N, I would modify fresh_threshold?
And for O, I would change max_order in the below area in cache_init.py?

    elif mode == 'Taylor':
        cache_dic['cache_type'] = 'random'
        cache_dic['cache_index'] = cache_index
        cache_dic['cache'] = cache
        cache_dic['fresh_ratio_schedule'] = 'ToCa' 
        cache_dic['fresh_ratio'] = 0.0
        cache_dic['fresh_threshold'] = 6
        cache_dic['force_fresh'] = 'global' 
        cache_dic['soft_fresh_weight'] = 0.0
        cache_dic['taylor_cache'] = True
        cache_dic['max_order'] = 1
        cache_dic['first_enhance'] = 3

Which means, the default values are N=6, O=1?

[Shenyi-Z]

Thank you for the quick response. To confirm, if I wanted to modify N, I would modify fresh_threshold? And for O, I would change max_order in the below area in cache_init.py?

    elif mode == 'Taylor':
        cache_dic['cache_type'] = 'random'
        cache_dic['cache_index'] = cache_index
        cache_dic['cache'] = cache
        cache_dic['fresh_ratio_schedule'] = 'ToCa' 
        cache_dic['fresh_ratio'] = 0.0
        cache_dic['fresh_threshold'] = 6
        cache_dic['force_fresh'] = 'global' 
        cache_dic['soft_fresh_weight'] = 0.0
        cache_dic['taylor_cache'] = True
        cache_dic['max_order'] = 1
        cache_dic['first_enhance'] = 3

Which means, the default values are N=6, O=1?

yes~

[Tammytcl]

Thanks for reporting the issue! 🙌

We’ve identified and fixed the circular import problem between layers.py and model.py, as well as the execution issue in cli_control.py. The SpeCa-FLUX codebase is now fully functional.

You can now run inference using the following command:

python -m flux.cli \
  --name flux-dev \
  --prompt "a misty forest, FLUX in red paint" \
  --width 1024 \
  --height 1024 \
  --num_steps 50 \
  --guidance 3.5 \
  --output_dir ./out_dev

Please pull the latest changes and let us know if you encounter any further issues!

Thanks for creating this combined repo!

It appears that the cache_cutfresh_by_cluster method is not defined anywhere, resulting in errors when attempting to use the ClusCa code.

In SpeCa, there is a circular import issue where layers.py and model.py both attempt to import each other, and cli_control.py also does not run.

When running diffusers_taylorseer_flux.py, with the command: python diffusers_taylorseer_flux.py --prompt "A sunset over a desert." I encounter the following error:

Traceback (most recent call last):
  File "Cache4Diffusion/flux/taylorseer_flux/diffusers_taylorseer_flux.py", line 136, in <module>
    main()
  File "Cache4Diffusion/flux/taylorseer_flux/diffusers_taylorseer_flux.py", line 110, in main
    image = pipeline(
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py", line 696, in __call__
    noise_pred = self.transformer(
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "Cache4Diffusion/flux/taylorseer_flux/forwards/flux_forward.py", line 110, in taylorseer_flux_forward
    image_rotary_emb = self.pos_embed(ids)
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 65, in forward
    [rope(ids[..., i], self.axes_dim[i], self.theta) for i in range(n_axes)],
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 65, in <listcomp>
    [rope(ids[..., i], self.axes_dim[i], self.theta) for i in range(n_axes)],
  File "Cache4Diffusion/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 44, in rope
    batch_size, seq_length = pos.shape
ValueError: not enough values to unpack (expected 2, got 1)

I am on diffusers v0.30.3. I did not see a requirements.txt file, is there a particular version I should use?

Thank you for the help!