Unable to achieve reported ScanNet performance with provided checkpoints

[chijw]

Thank you for open-sourcing your amazing work. However I encountered an issue when trying to reproduce the results from your paper. Specifically, using the provided checkpoint(downloaded from https://drive.google.com/drive/folders/18_FHE3-Hx_u3RV0K-8kzBP5xZGfgECQ3) and default settings in the code, the mIoU on ScanNet only reached about 0.4, which is noticeably lower than the 0.599 reported in the paper. Could you kindly clarify if I might be missing something? Also, is this the final version of the code used for the reported results?

[mayanheng123]

Hello, I am currently replicating this paper and reached the inference stage where a reference image is required. Could you please advise on the selection criteria for this reference image? Additionally, I would like to know the recommended GPU resources for efficiently processing the inference.

[chijw]

Hello, I am currently replicating this paper and reached the inference stage where a reference image is required. Could you please advise on the selection criteria for this reference image? Additionally, I would like to know the recommended GPU resources for efficiently processing the inference.

I just split the Replica and Scannet datasets into training and test sets according to the config files in the repo(but I am unsure if these are exactly the same datasets used in the paper, as the provided checkpoints do not achieve the reported performance). I ran the model on a single A6000 GPU, and it works fine. You can consider reducing the chunk_size to lower memory usage.