Robustness of 4DModel?

[dingmiaomiao]

Thank you for your work. I noticed that RoboBEV tested methods like BEVFormer using past frames. The RoboBEV dataset has only sample corruption. If the past frames are fine but the current frame is corrupted, it won’t reflect the true robustness of the 4D models. In real situations, both past and current frames should have the same corruption. How do you test the 4D model? Can you provide more details? Thanks!

[Daniel-xsy]

Thanks for your interest.

BEVFormer stores a history BEV feature inside the model, we follow the same evaluation protocol as testing on the original scenes dataset. Therefore, all the input (including past frames and current frames) are both corrupted data. This should reflect the real corruption situations.

It should be noted that since RoboBEV only corrupts sample data (not sweep), we use corrupted sample data as previous frames instead of using corrupted current sample and previous clean sweep data.