Skip to content

Disallow mlp_bias with MatFormer tiers (assert) #8

@plugyawn

Description

@plugyawn

NanoGPT MatFormer currently asserts if mlp_bias=true when matformer_tier>0. This avoids silently ignoring bias in sliced MLP paths. Decide whether to fully support bias in MatFormer (slice and add bias in helper/standard paths) or keep this restriction with documentation/config validation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions