Questions about the Pretrained Models

[jdk9405]

Hi, Thanks for sharing the nice work!

I’m curious about the differences between the pretrained models

For example,
BiFuseV2.pth : trained on Matterport3D-all
BiFuseV2_SpatialAudioGen.tph : trained on Matterport3D-all + tuned on SpatialAudioGen-all (p)
BiFuseV2_st3d.pth : trained on Matterport3D-all + tuned on Structured3D-all (p)

Is that correct?

[qiuzidian]

I am also curious about this question, the experiments in the paper confused me. using SpatialAudioGen's lots of unlabeled data seems to be not as good as Structured3D?

[albert100121]

Hi @jdk9405 ,

Thank you for liking our work!

Regarding the pretrained model checkpoints:

• BiFuseV2.pth: trained on Matterport3D-all
• BiFuseV2_SpatialAudioGen.pth: jointly trained on Matterport3D-all + SpatialAudioGen (p)
• BiFuseV2_st3d.pth: jointly trained on Matterport3D-all + Structured3D-all (p)

Where:

• p: pseudo label used
• all: the entire train/valid/test set is used

I wouldn't say it is "tuned," as they are jointly trained in all iterations.

[albert100121]

Hi @qiuzidian ,

Regarding the Stanford2D3D test set quantitative evaluation, the performance of the SpatialAudioGen dataset is slightly lower than that of the one jointly trained with Structured3D. However, the Structured3D dataset consists of indoor scenes that are highly similar to those in the Stanford2D3D test set, which explains this result. The main purpose of presenting the quantitative results is to demonstrate how the use of a large amount of pseudo-labeled data can enhance generalization ability.

It's also worth noting that the checkpoint trained with SpatialAudioGen outperforms the one trained with Structured3D in in-the-wild scenarios, both in terms of performance and generalization ability. The Structured3D dataset, being more focused on controlled indoor structures, offers less improvement for in-the-wild scenes.

Thanks for your interest in our work, and feel free to reach out if you have any more questions!

[qiuzidian]

Thank you for your answer!