Results

[ReefAlturki]

Dear authors,

I am trying to test the model and see the results, and I tried with different backbone networks [resnet18, resnet50, swin-t] but the results I get are different from the ones mentioned in the paper, for example I am getting a low track/MOTP score. I would appreciate your advice regarding this matter and thank you for your time!

Best wishes,

[wyhdata97]

How can i test the model?

[ReefAlturki]

Hi,
you just need to load the pre-trained models uploaded by the developers in this link: https://github.com/tteepe/EarlyBird/releases/tag/v1

unzip them and then write the following command: python main.py test -c model_weights/config.yaml
--ckpt model_weights/model-epoch=35-val_loss=6.50.ckpt

Could you please let me know if you get the same or close results to the papers? I am getting very low MOTP and MT scores and still trying to figure out why

Kind regards

[Graysonicc]

I also get low MOTP and MT scores. Have you found the cause of the problem？

[visionNoob]

Hi @ReefAlturki , @Graysonicc, @wyhdata97 ,

I am also experiencing low MOTP and MT scores when testing the model, similar to what you mentioned. Did you identify the cause of this issue or find a solution?

[visionNoob]

ahh I think I figured it out.

I've been digging into the low MOTP issue, and I just realized what’s going on.
In BEV, MOTP is calculated using Euclidean distance, not IOU.

Normally, higher MOTP = better tracking because it's based on IOU (where 1 is perfect).
But in BEV, we use Euclidean distance, and lower is better (closer to GT = better tracking).
So MOTP is inverted, making it look artificially low when using Euclidean distance.

we just flip the MOTP value like this:

summary["motp"] = 1 - summary["motp"]