Evaluation on waymo opendataset

[QingXIA233]

Hello, in order to reproduce the waymo results by my own, I trained CenterFormer on waymo and tried to get the performance evaluation like that shown in the README:

I follwed the instruction from https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/quick_start.md, I already have the gt.bin and preds.bin of CenterFormer, but I ran into this error:

I wonder whether you encounted this issue before, or maybe I've gone to the wrong way? Really need some help here. Please. Thanks in advance.

[DezeZhao]

hello, have you evaluated waymo with det3d code? can you reproduce the results?

[QingXIA233]

As I see in the original Det3D repo, https://github.com/poodarchu/Det3D/blob/master/GETTING_STARTED.md. The evaluation is for Nuscenes, lyft and kitti, not for waymo. What my question is: after I train CenterFormer on waymo and get the checkpoint, how could I get the level1 and level2 MAP evaluation results like you have shown in your repo.

[DezeZhao]

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

[edwardzhou130]

Hello, in order to reproduce the waymo results by my own, I trained CenterFormer on waymo and tried to get the performance evaluation like that shown in the README: I follwed the instruction from https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/quick_start.md, I already have the gt.bin and preds.bin of CenterFormer, but I ran into this error: I wonder whether you encounted this issue before, or maybe I've gone to the wrong way? Really need some help here. Please. Thanks in advance.

It's possible that the Waymo open dataset API wasn't built correctly. You can use their fake pred and gt to test first.

[edwardzhou130]

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

I haven't previously worked on KITTI. But I believe that due to the significant differences in point cloud style between WOD/nuscenes and KITTI, you will need to make significant changes to the training configs. Can you provide more information about how you've used the code and set the configs?

[DezeZhao]

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

I haven't previously worked on KITTI. But I believe that due to the significant differences in point cloud style between WOD/nuscenes and KITTI, you will need to make significant changes to the training configs. Can you provide more information about how you've used the code and set the configs?

ok thank you for your reply. I can show you the change of code later. canyou help me about another issue about the waymo gt boxes xyzwlhr. why r translate to -r-pi/2. Does this mean translate to camera coordinate? but waht I understand is that it is no need.

[zhaowenZhou]

Hi, did you solve it?

[zhaowenZhou]

oh, yeah, I got it. I also met a problem that I trained CenterFormer on kitti, but I found it showed very low performance. I also trained it on waymo, but found not equal results as paper shows. tips: I trained them on OpenPCDet, as I transplant the code to the generally used repo OpenPCDet. If you have interest in it, I can share with you. If possible, we can Communicate with each other about some 3D detection related question, we can chat with wechat: 17717078595.

你好，我搜不到你的微信，请问怎么联系你？