About the evaluation metric mAP

[Zhong1015]

Hello, your code has been incredibly beneficial to me. Regarding the mAP evaluation metric you used, I wanted to confirm if the calculation is accurate. There seems to be limited information online about the retrieval_average_precision function that you used for mAP calculation. Upon verification, it appears that the rAP calculation you provided might differ from the typical mAP calculation.

import torch
from torch import Tensor, tensor
from torchmetrics.classification import AveragePrecision
from torchmetrics.utilities.checks import _check_retrieval_functional_inputs

def retrieval_average_precision(preds: Tensor, target: Tensor) -> Tensor:

# 检查preds和target的shape是否正确，错误则返回Error
preds, target = _check_retrieval_functional_inputs(preds, target)

# 检查真实标签是否包含全0的情况，也就是在真实标签中，针对一个preds和targets对有没有真实标签全为0的情况
if not target.sum():
    return tensor(0.0, device=preds.device)

# 根据preds的维度对target张量进行排序(按照最内部的维度，降序排序)
target = target[torch.argsort(preds, dim=-1, descending=True)]
# 得到target＞0，也就是真实标签为正样本对应的target位置
positions = torch.arange(1, len(target) + 1, device=target.device, dtype=torch.float32)[target > 0]

res = torch.div((torch.arange(len(positions), device=positions.device, dtype=torch.float32) + 1), positions).mean()
return res

import torch

preds = torch.tensor([0.2, 0.3, 0.4,0.6, 0.5, 0.5,0.4, 0.3, 0.2,0.6, 0.4, 0.5,0.7, 0.7, 0.4])
labels = torch.tensor([True, False, True, True, True, False,False, False, True,False, True, True,True, False, True])

计算平均精度

ap = retrieval_average_precision(preds, labels)
print(ap.item()) # 打印平均精度值

##############################################################################################
from torch import tensor

average_precision = AveragePrecision(task="binary")
x=average_precision(preds, labels)
print(x)

##############################################################################################
import torch
from sklearn.metrics import average_precision_score
ap = average_precision_score(labels, preds)

print("Average Precision (AP):", ap)

The results are:
0.6523208022117615
tensor(0.5835)
Average Precision (AP): 0.5835497835497836

The first one was calculated using the retrieval average precision method you provided, the second one was obtained using the AP calculation within Torchmetrics for classification, and the third one was computed using the AP calculation within sklearn. The result from the first method differs from the results obtained from the second and third methods.