MPSA (Mixed Precision Sparse Attention)

A training free sparse attention implementation. report.

Requirements

• torch >= 2.9.0

Installation

pip install -e .

Performance (0.8 sparsity, TFlops, 4090)

Sparse TFlops: calculated by full attention formula.

Sage Attn 2++	MPSA Sparse Attn (0.8)
440 TFlops	1320 TFlops

Example

• simple wan 2.2 14B end-to-end example: python example.py

• benchmark sparse attn: python test/test_mpsa.py

Code

• Sage Attention 2++ Gluon DSL: Folder

• sparse attention Gluon DSL: File

Sample Videos

Using Wan 2.2 14B model.

Single Image to Video

Full Attn	Sparse Attn

First-and-Last Image to Video

Full Attn	Sparse Attn