Speed of linear attention slower than the attention implemented in pytorch #130
Description
I was testing the speed of linear attention, but found that its speed is slower than the standard attention implemented in pytorch. Could you help me to understand this? Is this because the pytorch implementation include flash attention?