Baselines That Are Deliberately Lowered #3
Description
I am replying to my previous issue because it was directly closed by @xiaoaoran.
@xiaoaoran I am afraid that your explanations are not convincing or resolve any concerns.
- You got good-looking numbers but your comparisons are unfair. You used hardware to explain the low baseline scores but it should not be a valid excuse. Let me give you a simple example. Suppose that people are obtaining 58.23 NDS using PointPillar in OpenPCDet, why would I trust your results (55.7 improved from 54.9) that are lowered because of the number of GPUs used? Note that hundreds of researchers are able to reproduce the 58.23 NDS. If the hardware could become an excuse, then anybody can claim that their baseline is low because they don't have 4 GPUs, which is ridiculous.
- You are listing facts that Cylinder3D, 2DPASS, and PVKD are having much higher scores than you. But you are not responding to why you choose methods that scores are behind the current trend as your baseline. To me, you are just picking old baselines and adding up something on them. The real proof that an augmentation should be implemented on the current state-of-the-art is never verified by you, which leads to the key problem of your work.
- I am really curious why the reviewers of NeurIPS have never pointed out these issues for you.
A valid proof requires redos on the normal baselines.