Dataset Size on Leaderboard

[brunnedu]

Hey there,

While reviewing the leaderboard submissions for the small filtering track, I observed instances where the dataset size was noted as 1.3e7, which is essentially equivalent to the original dataset size without any filtering. Now I was wondering whether it was actually the case that these submissions kept almost all of the original data (which I doubt given submission titles such as BLIP2-COCO-finetuned_similarity_top-35%) or if this is an error.

Thanks for clarifying!

[gabrielilharco]

Hi @brunnedu. Thanks for the comment! My best guess is that the authors from those submissions forgot to correctly pass the flag specifying the dataset size