Skip to content
This repository was archived by the owner on Nov 16, 2023. It is now read-only.

Docs for Multiclass Classification Metrics are wrong. #501

@antoniovs1029

Description

@antoniovs1029

The docs on https://docs.microsoft.com/en-us/nimbusml/concepts/metrics#multiclass-classification-metrics are wrong since it states the following:

This corresponds to evaltype=’multiclass’.

[...]

(class N) - Accuracy of class N

Problem is that the (class N) columns that are the output of the evaluation aren't Per class Accuracies but actually Per class Log Loss metrics.

I've stepped into from ML.NET to NimbusML and this was the case.

For instance, ML.NET's MulticlassClassificationMetrics object doesn't hold a PerClassAccuracy member, but it does PerClassLogLoss

https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.data.multiclassclassificationmetrics?view=ml-dotnet

And in ML.NET's multiclass evaluator there's no code to calculate per class accuracy, only per class log loss:
https://github.com/dotnet/machinelearning/blob/c255ac7c45af7a19a6e5ab1e26fdbe44874f241a/src/Microsoft.ML.Data/Evaluators/MulticlassClassificationEvaluator.cs#L298-L308

So I don't know if maybe in the past ML.NET/NimbusML actually output per class accuracies, but at least now it does seem the docs are wrong and would need to be fixed. Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions