Skip to content

Update Metric interface to return None for no-value #968

@mdekstrand

Description

@mdekstrand

Metrics should perhaps explicitly return None when they cannot compute a value (e.g., no truth), and NaN only when there is floating-point problems.

Note

We should not return None when the recommendation list itself is empty — an empty list has no utility.

Reasons for:

  • Clearly distinguish "we cannot score this list, and no model could do better" from other floating-point problems. Drop no-truth items from aggregates.

Reasons against:

  • No-truth cases are well-defined for many metrics (e.g., 0 RBP or RR). For NDCG they are undefined (divide by 0), but NaN is a reasonable non-error return for a division by zero.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Status

    Backlog

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions