Adapting to VIT S

[AnielAlexa]

Hello, do you have some guides how to adapt this to VIT S? It produce 384 dim vs 1024 VIT L so Multi-view Transformer need to be 2x384->768->1024. I'm interested in that for deploying on edge devices like Jetson. Thank you!

[HustBestCat]

I’d actually recommend aligning everything to the smaller dimension instead of upsampling to 1024. The features from a pretrained ViT-S are already quite powerful, and keeping the dimension low greatly benefits edge deployment. You can redesign the multi-view fusion module to operate in 384-D space, then fine-tune the whole pipeline from scratch or with partial weight initialization. This approach is much more practical for Jetson.

[AnielAlexa]

But i do not have the training script :((

[AnielAlexa]

@Parskatt