Runtime Error

[swingler]

Compile and deploy successful, but runtime error:
[ComfyUI-Manager] All startup tasks have been completed.
[01:58:53] [ 51.12s] [ModelLoader] Cleaned 1575 state dict keys
[01:58:53] [ 51.12s] [ModelLoader] Applying quantization-aware replacements (quant_linear=True, fast_norm=True)...
[01:58:53] [ 51.14s] [ModelLoader] Loading weights into model...
[01:58:53] [ 51.16s] [ModelLoader] Model loaded to CPU
[01:58:53] [ 51.17s] [ModelLoader] Model wrapped with ComfyUI-native async offloading
[01:58:53] [ 51.17s] [ModelLoader] ✓ Successfully loaded model
[01:58:53] [ 51.17s] [ModelLoader] Model type: Wan2.2-A14B
[01:58:53] [ 51.17s] [ModelLoader] Attention: sla
[01:58:53] [ 51.17s] [ModelLoader] Quantized: True
[01:58:53] [ 30.91s] ✓ Model loaded in 30.91s: TurboWan2.2-I2V-A14B-high-720P-quant.pth
[01:58:53] [ 34.57s] [I2V-Inference] VRAM after high noise model loaded: 0.01GB allocated, 0.09GB reserved, 11.54GB free
[01:58:53] [ 34.57s] [I2V-Inference] Sampling with high noise model (steps 0-3)...
Sampling steps 0-3: 0%| | 0/3 [00:00<?, ?it/s]Requested to load WanModel
loaded completely; 10062.65 MB usable, 445.90 MB loaded, full load: False
terminate called after throwing an instance of 'c10::AcceleratorError'
what(): CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at /pytorch/c10/cuda/CUDAException.cpp:42 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x80 (0x7fbb17ad9eb0 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10.so)
frame #1: + 0x111c7 (0x7fbb17b6c1c7 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #2: + 0x560b9 (0x7fbb17bb10b9 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #3: + 0x56b64 (0x7fbb17bb1b64 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #4: + 0x434a5f (0x7fbb0a434a5f in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_python.so)
frame #5: c10::TensorImpl::~TensorImpl() + 0x9 (0x7fbb17ab7179 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10.so)
frame #6: + 0xf01fda (0x7fbaf6901fda in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #7: + 0x2aed68c (0x7fbaf84ed68c in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #8: at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, std::optionalc10::ScalarType, std::optionalc10::Layout, std::optionalc10::Device, std::optional, bool, std::optionalc10::MemoryFormat) + 0xf2 (0x7fbaf7bc34d2 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #9: + 0x28ba68b (0x7fbaf82ba68b in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #10: at::_ops::_to_copy::call(at::Tensor const&, std::optionalc10::ScalarType, std::optionalc10::Layout, std::optionalc10::Device, std::optional, bool, std::optionalc10::MemoryFormat) + 0x23d (0x7fbaf7c64f7d in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #11: at::native::to(at::Tensor const&, c10::Device, c10::ScalarType, bool, bool, std::optionalc10::MemoryFormat) + 0x9b (0x7fbaf7710b4b in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #12: + 0x2be8818 (0x7fbaf85e8818 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #13: at::_ops::to_device::call(at::Tensor const&, c10::Device, c10::ScalarType, bool, bool, std::optionalc10::MemoryFormat) + 0x1c9 (0x7fbaf7dfc309 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #14: + 0x424d55 (0x7fbb0a424d55 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_python.so)
frame #15: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x546399]
frame #16: PyObject_Vectorcall + 0x51 (0x538e71 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #17: _PyEval_EvalFrameDefault + 0x6d0 (0x521440 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #18: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #19: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #20: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #21: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #22: _PyObject_FastCallDictTstate + 0x1e7 (0x519a47 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #23: _PyObject_Call_Prepend + 0x66 (0x5529c6 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #24: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x627356]
frame #25: _PyObject_MakeTpCall + 0x2fc (0x5172ac in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #26: _PyEval_EvalFrameDefault + 0x6d0 (0x521440 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #27: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #28: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #29: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #30: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #31: _PyObject_FastCallDictTstate + 0x1e7 (0x519a47 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #32: _PyObject_Call_Prepend + 0x66 (0x5529c6 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #33: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x627356]
frame #34: _PyObject_MakeTpCall + 0x2fc (0x5172ac in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #35: _PyEval_EvalFrameDefault + 0x6d0 (0x521440 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #36: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #37: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #38: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #39: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #40: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #41: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #42: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #43: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #44: _PyObject_FastCallDictTstate + 0x285 (0x519ae5 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #45: _PyObject_Call_Prepend + 0x66 (0x5529c6 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #46: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x627356]
frame #47: _PyObject_Call + 0xb5 (0x555605 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #48: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #49: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #50: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x62ba4f]
frame #51: _PyObject_MakeTpCall + 0x2fc (0x5172ac in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #52: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #53: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #54: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #55: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #56: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #57: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #58: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #59: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #60: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #61: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #62: _PyObject_FastCallDictTstate + 0x285 (0x519ae5 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)

[zoldaten]

i have a problem also with:
python turbodiffusion/inference/wan2.1_t2v_infer.py --model Wan2.1-1.3B --dit_path checkpoints/TurboWan2.1-T2V-1.3B-480P-quant.pth --resolution 480p --prompt "A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about." --num_samples 1 --num_steps 4 --quant_linear --attention_type sla --sla_topk 0.1

Megatron-core is not installed.
[12-30 22:04:45|INFO|turbodiffusion\inference\wan2.1_t2v_infer.py:74:<module>] Computing embedding for prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
[12-30 22:04:45|INFO|turbodiffusion\rcm\utils\umt5.py:495:__init__] loading checkpoints/models_t5_umt5-xxl-enc-bf16.pth
C:\Users\Al\turbod\lib\site-packages\torch\_utils.py:99: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:28.)
  untyped_storage = torch.UntypedStorage(self.size(), device=device)
[12-30 22:04:54|INFO|turbodiffusion\inference\wan2.1_t2v_infer.py:79:<module>] Loading DiT model from checkpoints/TurboWan2.1-T2V-1.3B-480P-quant.pth
[12-30 22:04:54|INFO|turbodiffusion\rcm\networks\wan2pt1.py:802:enable_selective_checkpoint] Enable selective checkpoint with mm_only, for every 1 blocks. Total blocks: 30
[12-30 22:04:55|SUCCESS|turbodiffusion\inference\wan2.1_t2v_infer.py:82:<module>] Successfully loaded DiT model.
[12-30 22:04:56|INFO|turbodiffusion\rcm\tokenizers\wan2pt1.py:590:_video_vae] loading checkpoints/Wan2.1_VAE.pth
[12-30 22:04:56|INFO|turbodiffusion\inference\wan2.1_t2v_infer.py:88:<module>] Generating with prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
Sampling:   0%|                                                                                                                | 0/4 [00:00<?, ?it/s]C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Memory efficient kernel not used because: (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:776.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Memory Efficient attention has been runtime disabled. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen/native/transformers/sdp_utils_cpp.h:551.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Flash attention kernel not used because: (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:778.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Torch was not compiled with flash attention. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:598.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: CuDNN attention kernel not used because: (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:780.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: CuDNN attention has been runtime disabled. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:528.)
  out = torch.nn.functional.scaled_dot_product_attention(
Sampling:   0%|                                                                                                                | 0/4 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\inference\wan2.1_t2v_infer.py", line 131, in <module>
    v_pred = net(x_B_C_T_H_W=x.to(**tensor_kwargs), timesteps_B_T=(t_cur.float() * ones * 1000).to(**tensor_kwargs), **condition).to(
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 698, in forward
    x_B_L_D = block(x_B_L_D, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\\turbod\lib\site-packages\torch\distributed\algorithms\_checkpoint\checkpoint_wrapper.py", line 170, in forward
    return self.checkpoint_fn(  # type: ignore[misc]
  File "C:\Users\Al\turbod\lib\site-packages\torch\_compile.py", line 32, in inner
    return disable_fn(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\_dynamo\eval_frame.py", line 745, in _fn
    return fn(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\utils\checkpoint.py", line 496, in checkpoint
    ret = function(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 416, in forward
    x = cross_attn_ffn(x, context, context_lens, e)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 410, in cross_attn_ffn
    x = x + self.cross_attn(self.norm3(x), context, context_lens)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 296, in forward
    x = self.attn_op(q, k, v)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\a2a_cp.py", line 199, in forward
    results = super().forward(query, key, value, *args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\a2a_cp.py", line 173, in forward
    return self.local_attn(query, key, value, *args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py", line 162, in attention
    out = torch.nn.functional.scaled_dot_product_attention(
RuntimeError: No available kernel. Aborting execution.

It seems it tries to sampling twice and crash. i`m on Windows 10.

pip:

antlr4-python3-runtime==4.9.3
attrs==25.4.0
certifi==2025.11.12
charset-normalizer==3.4.4
colorama==0.4.6
einops==0.8.1
exceptiongroup==1.3.1
filelock==3.20.0
flash_attn @ file:///C:/Users/Al/Downloads/flash_attn-2.7.4%252Bcu126torch2.6.0cxx11abiFALSE-cp310-cp310-win_amd64.whl#sha256=dad211664253bd8298bdb8493178f86a4ed0e81500b69655a86726c3d19555ad
fsspec==2025.12.0
ftfy==6.3.1
fvcore==0.1.5.post20221221
huggingface-hub==0.36.0
idna==3.11
ImageIO==2.37.2
imageio-ffmpeg==0.6.0
iniconfig==2.3.0
iopath==0.1.10
Jinja2==3.1.6
loguru==0.7.3
markdown-it-py==4.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
-e git+https://github.com/NVIDIA/Megatron-LM.git@dd7c9f4f6963e133a07515ffc51fa331a8658184#egg=megatron_core
mpmath==1.3.0
networkx==3.4.2
ninja==1.13.0
numpy==2.2.6
nvidia-ml-py==13.590.44
omegaconf==2.3.0
packaging==25.0
pandas==2.3.3
pillow==12.0.0
pluggy==1.6.0
portalocker==3.2.0
prompt_toolkit==3.0.52
psutil==7.2.1
Pygments==2.19.2
pytest==9.0.2
python-dateutil==2.9.0.post0
pytz==2025.2
pywin32==311
PyYAML==6.0.3
regex==2025.11.3
requests==2.32.5
rich==14.2.0
safetensors==0.7.0
six==1.17.0
sympy==1.13.1
tabulate==0.9.0
termcolor==3.3.0
tokenizers==0.22.1
tomli==2.3.0
torch==2.6.0+cu126
torchaudio==2.6.0+cu126
torchvision==0.21.0+cu126
tqdm==4.67.1
transformers==4.57.3
triton-windows==3.5.1.post23
-e git+https://github.com/thu-ml/TurboDiffusion.git@50165f071ff476433f923244ad0ba0fd9e0c9d92#egg=turbodiffusion
typing_extensions==4.15.0
tzdata==2025.3
urllib3==2.6.2
wcwidth==0.2.14
win32_setctime==1.2.0
yacs==0.1.8

[zoldaten]

p.s. i tried with torch==2.8.0+cu126 and tried to change *sla, *original arguments but no result.

original in args gives this:

  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 2624, in load_state_dict
    raise RuntimeError(
RuntimeError: Error(s) in loading state_dict for WanModel:
        Unexpected key(s) in state_dict: "blocks.0._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.0._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.1._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.1._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.2._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.2._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.3._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.3._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.4._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.4._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.5._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.5._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.6._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.6._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.7._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.7._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.8._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.8._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.9._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.9._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.10._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.10._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.11._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.11._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.12._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.12._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.13._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.13._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.14._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.14._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.15._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.15._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.16._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.16._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.17._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.17._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.18._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.18._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.19._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.19._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.20._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.20._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.21._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.21._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.22._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.22._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.23._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.23._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.24._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.24._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.25._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.25._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.26._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.26._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.27._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.27._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.28._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.28._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias", "blocks.29._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.weight", "blocks.29._checkpoint_wrapped_module.self_attn.attn_op.local_attn.proj_l.bias".

i also check flash_atten available:

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.__version__)
2.8.0+cu126
>>> print("CUDA Version: ", torch.version.cuda)
CUDA Version:  12.6
>>> print("Device name:", torch.cuda.get_device_properties("cuda").name)
Device name: NVIDIA GeForce RTX 3060
>>> print("FlashAttention available:", torch.backends.cuda.flash_sdp_enabled())
FlashAttention available: True

[Explorer-Dong]

i have a problem also with: python turbodiffusion/inference/wan2.1_t2v_infer.py --model Wan2.1-1.3B --dit_path checkpoints/TurboWan2.1-T2V-1.3B-480P-quant.pth --resolution 480p --prompt "A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about." --num_samples 1 --num_steps 4 --quant_linear --attention_type sla --sla_topk 0.1

Megatron-core is not installed.
[12-30 22:04:45|INFO|turbodiffusion\inference\wan2.1_t2v_infer.py:74:<module>] Computing embedding for prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
[12-30 22:04:45|INFO|turbodiffusion\rcm\utils\umt5.py:495:__init__] loading checkpoints/models_t5_umt5-xxl-enc-bf16.pth
C:\Users\Al\turbod\lib\site-packages\torch\_utils.py:99: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:28.)
  untyped_storage = torch.UntypedStorage(self.size(), device=device)
[12-30 22:04:54|INFO|turbodiffusion\inference\wan2.1_t2v_infer.py:79:<module>] Loading DiT model from checkpoints/TurboWan2.1-T2V-1.3B-480P-quant.pth
[12-30 22:04:54|INFO|turbodiffusion\rcm\networks\wan2pt1.py:802:enable_selective_checkpoint] Enable selective checkpoint with mm_only, for every 1 blocks. Total blocks: 30
[12-30 22:04:55|SUCCESS|turbodiffusion\inference\wan2.1_t2v_infer.py:82:<module>] Successfully loaded DiT model.
[12-30 22:04:56|INFO|turbodiffusion\rcm\tokenizers\wan2pt1.py:590:_video_vae] loading checkpoints/Wan2.1_VAE.pth
[12-30 22:04:56|INFO|turbodiffusion\inference\wan2.1_t2v_infer.py:88:<module>] Generating with prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
Sampling:   0%|                                                                                                                | 0/4 [00:00<?, ?it/s]C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Memory efficient kernel not used because: (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:776.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Memory Efficient attention has been runtime disabled. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen/native/transformers/sdp_utils_cpp.h:551.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Flash attention kernel not used because: (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:778.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: Torch was not compiled with flash attention. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:598.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: CuDNN attention kernel not used because: (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:780.)
  out = torch.nn.functional.scaled_dot_product_attention(
C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py:162: UserWarning: CuDNN attention has been runtime disabled. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:528.)
  out = torch.nn.functional.scaled_dot_product_attention(
Sampling:   0%|                                                                                                                | 0/4 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\inference\wan2.1_t2v_infer.py", line 131, in <module>
    v_pred = net(x_B_C_T_H_W=x.to(**tensor_kwargs), timesteps_B_T=(t_cur.float() * ones * 1000).to(**tensor_kwargs), **condition).to(
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 698, in forward
    x_B_L_D = block(x_B_L_D, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\\turbod\lib\site-packages\torch\distributed\algorithms\_checkpoint\checkpoint_wrapper.py", line 170, in forward
    return self.checkpoint_fn(  # type: ignore[misc]
  File "C:\Users\Al\turbod\lib\site-packages\torch\_compile.py", line 32, in inner
    return disable_fn(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\_dynamo\eval_frame.py", line 745, in _fn
    return fn(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\utils\checkpoint.py", line 496, in checkpoint
    ret = function(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 416, in forward
    x = cross_attn_ffn(x, context, context_lens, e)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 410, in cross_attn_ffn
    x = x + self.cross_attn(self.norm3(x), context, context_lens)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\networks\wan2pt1.py", line 296, in forward
    x = self.attn_op(q, k, v)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\Al\turbod\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\a2a_cp.py", line 199, in forward
    results = super().forward(query, key, value, *args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\a2a_cp.py", line 173, in forward
    return self.local_attn(query, key, value, *args, **kwargs)
  File "C:\Users\Al\TurboDiffusion\turbodiffusion\rcm\utils\attention.py", line 162, in attention
    out = torch.nn.functional.scaled_dot_product_attention(
RuntimeError: No available kernel. Aborting execution.

It seems it tries to sampling twice and crash. i`m on Windows 10.

pip:

antlr4-python3-runtime==4.9.3
attrs==25.4.0
certifi==2025.11.12
charset-normalizer==3.4.4
colorama==0.4.6
einops==0.8.1
exceptiongroup==1.3.1
filelock==3.20.0
flash_attn @ file:///C:/Users/Al/Downloads/flash_attn-2.7.4%252Bcu126torch2.6.0cxx11abiFALSE-cp310-cp310-win_amd64.whl#sha256=dad211664253bd8298bdb8493178f86a4ed0e81500b69655a86726c3d19555ad
fsspec==2025.12.0
ftfy==6.3.1
fvcore==0.1.5.post20221221
huggingface-hub==0.36.0
idna==3.11
ImageIO==2.37.2
imageio-ffmpeg==0.6.0
iniconfig==2.3.0
iopath==0.1.10
Jinja2==3.1.6
loguru==0.7.3
markdown-it-py==4.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
-e git+https://github.com/NVIDIA/Megatron-LM.git@dd7c9f4f6963e133a07515ffc51fa331a8658184#egg=megatron_core
mpmath==1.3.0
networkx==3.4.2
ninja==1.13.0
numpy==2.2.6
nvidia-ml-py==13.590.44
omegaconf==2.3.0
packaging==25.0
pandas==2.3.3
pillow==12.0.0
pluggy==1.6.0
portalocker==3.2.0
prompt_toolkit==3.0.52
psutil==7.2.1
Pygments==2.19.2
pytest==9.0.2
python-dateutil==2.9.0.post0
pytz==2025.2
pywin32==311
PyYAML==6.0.3
regex==2025.11.3
requests==2.32.5
rich==14.2.0
safetensors==0.7.0
six==1.17.0
sympy==1.13.1
tabulate==0.9.0
termcolor==3.3.0
tokenizers==0.22.1
tomli==2.3.0
torch==2.6.0+cu126
torchaudio==2.6.0+cu126
torchvision==0.21.0+cu126
tqdm==4.67.1
transformers==4.57.3
triton-windows==3.5.1.post23
-e git+https://github.com/thu-ml/TurboDiffusion.git@50165f071ff476433f923244ad0ba0fd9e0c9d92#egg=turbodiffusion
typing_extensions==4.15.0
tzdata==2025.3
urllib3==2.6.2
wcwidth==0.2.14
win32_setctime==1.2.0
yacs==0.1.8

When you use sla attention type, you need to install SpargeAttention first:

pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation