Skip to content

Wire helper-mode sparse indices into DisTrO aggregation #4

@plugyawn

Description

@plugyawn

Problem

MatFormer helper mode includes align_matformer_sparse_grad, but it’s not wired into DisTrO or the network payloads. Helper indices are not transmitted, and the aggregation path only aligns contiguous prefix grads.

Refs:

  • shared/modeling/src/distro.rs (TODO added above align_matformer_sparse_grad)

Expected

If helper mode uses sparse index selection, the receiver must scatter into full shapes for aggregation.

Possible Approach

  • Extend DistroResult/payloads to include helper indices (or a reproducible seed/epoch for reconstruction).
  • Use align_matformer_sparse_grad in the hetero-aggregation path.
  • Add tests for helper-mode sparse aggregation.

Acceptance Criteria

  • Helper mode works with sparse gradient transmission.
  • Aggregation produces correct full-shape grads.
  • Tests validate helper-mode wiring end-to-end.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions