About training data for ScanNet

[swwddjjp]

Dear authors, I am very interested in your research and am trying to reproduce code.
Now I have some questions about experiments on ScanNet dataset. You said you have 4357 chairs and 1271 tables are extracted as the training set, while the validation set contains 1368 chairs and 350 tables. Howerver, the preprocessed data you provided seems only contain validation set.
Additionally, my reproduced results on the EPN3D dataset align closely with those reported in your paper. However, when training and testing on the ScanNet validation set, I observed significant discrepancies in the outcomes. I'm trying to determine whether this stems from:

1. The absence of training data usage,
2. Potential miscalculations related to the unidirectional nature of evaluation metrics (UCD, UHD, RCD), where I followed your code implementation (e.g., UCD(ret,gt)). Could the partial GT characteristics in ScanNet necessitate reversing the calculation direction to UCD(gt,ret) instead?

[CuiRuikai]

Thank you very much for your interest.

It’s weird that you have significant difference. After all, if we have a powerful neural network that does purely autoencoding (outputs just the input objects), we can even reduce the UCD and UHD numbers to zero.

I have some thoughts about possible cause: Have you normalized the shapes before evaluation? Or you may use a wrong normalization method? For example, normalize to a cube or to a sphere, to the [-1, 1] or [0, 1] scale. I did apply some kind of normalization, but I have forgotten what is it.

Another possible cause might be the absence of training data, as you mentioned. The way you do the object extraction may be different from mine, leading to some domain gap between your training set and my validation set. Therefore, I suggest you to extract your own validation set and verify if the metrics become reasonable.