-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Problem
MatFormer helper mode includes align_matformer_sparse_grad, but it’s not wired into DisTrO or the network payloads. Helper indices are not transmitted, and the aggregation path only aligns contiguous prefix grads.
Refs:
shared/modeling/src/distro.rs(TODO added abovealign_matformer_sparse_grad)
Expected
If helper mode uses sparse index selection, the receiver must scatter into full shapes for aggregation.
Possible Approach
- Extend
DistroResult/payloads to include helper indices (or a reproducible seed/epoch for reconstruction). - Use
align_matformer_sparse_gradin the hetero-aggregation path. - Add tests for helper-mode sparse aggregation.
Acceptance Criteria
- Helper mode works with sparse gradient transmission.
- Aggregation produces correct full-shape grads.
- Tests validate helper-mode wiring end-to-end.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels