Mismatch between the code and the uploaded checkpoint

[Peter-SungwooCho]

Hello! Thank you for sharing such an impressive work!

After reviewing the code, I was able to reproduce most of the results successfully. However, I noticed two issues and wanted to report them here.

1. The crema_v2 code referenced in the scripts is missing.

2. While the provided crema code in the current repo runs after fixing some minor bugs, loading the checkpoint during both training and inference results in too many missing and unexpected keys.

My impression is that the backbones used in crema v1 and v2 are different — it seems that the code corresponds to v1, while the uploaded checkpoint is from v2. Could you solve this problem?

Best regards,
Sungwoo Cho

[Yui010206]

Hi Sungwoo, thanks for your interest in CREMA project. We just found and fixed the saving ckpt bug (since squential training makes only one modality ckpt unfrozen), please check the new code.
And we also find current v2 ckpts are incomplete, we will launch the new experiments and upload ckpt later.