Skip to content

Negative variance iff edge ablations in Split LN #318

@stefan-apollo

Description

@stefan-apollo

Because of our RIB rotations, the variance node can experience arbitrary changes, including becoming negative.

"Fixes" include var = torch.abs(var). (This is actually fine -- if var becomes negative then even abs(var) is gonna be a nonsense value and the loss is gonna be terrible still.)

Think about what is least terrible and implement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions