Skip to content

Support bwd packgqa for ffa#235

Open
hanwen-sun wants to merge 15 commits intomainfrom
support_bwd_packgqa
Open

Support bwd packgqa for ffa#235
hanwen-sun wants to merge 15 commits intomainfrom
support_bwd_packgqa

Conversation

@hanwen-sun
Copy link
Copy Markdown
Collaborator

@hanwen-sun hanwen-sun commented Feb 3, 2026 •
edited
Loading

what this pr do:

  • enable packgqa for bwd when swapbwdqkloop is true.
  • write 0 back to log2lse and dpsum for invalid index.
  • support packgqa for bwd when swapbwdqkloop is false.

TODO in feature:

  • test the performance when autorangemerge is ready.
  • test performance for packgqa with swapbwdqkloop false.
  • support deterministc for packgqa bwd.

@hanwen-sun hanwen-sun changed the title [WIP] Support bwd packgqa Support bwd packgqa with swap_qk_loop Feb 4, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@KevinZeng08
Copy link
Copy Markdown
Contributor

KevinZeng08 commented Feb 4, 2026

range merge is implemented in #240

@hanwen-sun hanwen-sun changed the title Support bwd packgqa with swap_qk_loop Support bwd packgqa for ffa Feb 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hanwen-sun @KevinZeng08