Evaluation using pre-trained weights yielded much better results than reported results

[caoyiyang1105]

I conducted an evaluation of the model using the pre-trained weights (humanml_trans_dec_512_bert, 50 steps) on the HumanML3D dataset. The results significantly outperformed the reported figures (my evaluation result is as follows.).

Furthermore, I observed that in the evaluation file (eval_humanml_humanml_trans_dec_bert_512_000600000_gscale2.5.log), located in the same folder as the pre-training weights, the calculated metrics (R-Precision, MM-Dist, Diversity) deviate considerably from the accepted ground truth has a big deviation. Could you please elaborate on the specific reasons behind this?

I would like to express my sincere gratitude for your assistance !!!

[GuyTevet]

Regarding your BERT model evaluation, that's interesting! What CFG did you use?

Regarding the GT performance, I see as well that the matching score is higher and the precision is slightly lower compared to the original evaluation from two years ago.
I would like to understand if this is a data issue or a code issue. As far as I know, the data and the evaluation checkpoints were not changed on my end.
Can you try to revert the code to a commit from the early days and evaluate the original model there (to check GT performance)?