wrong random F1 score

[hsv-tse-kun]

I use LLaVA-1.5-7B to run the pope benchmark, but its random samples testing score do not match the theoretical score, could you please help me with the problem?

[peterwisu]

I got a similar problem with the variant of LLava. Have you solve this issue?

[1jia1-oneplusone]

You can see the "# samples" of random samples is 2910, while 3000 is the correct number. You miss some samples in the middle of the random subset, therefore the subsequent samples are in the wrong places, not aligning with the labels.