NSA Support

[jensenzhoujh]

Hi,

Thanks for the great work! I saw in the roadmap that NSA will be supported. Do potentially have a timeline for this feature?

Best,

[littsk]

Hello,
thank you for your interest in Magi-Attention! We are excited to announce that support for NSA will be released within the next 2 to 3 weeks. Stay tuned for updates and new features.
Your support means a lot to us!

[jensenzhoujh]

Great thanks! Looking forward to the implementation!

I am interested in utilizing repo for ultra-long sequence modeling but I find current repo does have a lot examples for this scenario. It would be tremendously helpful if some toy examples can be provided for long-context modeling with some seminar works (e.g., strided attention (Sparse Attention), dilated attention (LongNet) and NSA, etc.)

Best,