Handle different resolution in the pair of views

[yindaheng98]

If pair of views has different resolution (this is common after undistortion by colmap), size of msk comes from self.conf_i[i_j], while size of self.pred_j[i_j] comes from the other view, cause size mismatch error in fast_pnp (at pts3d[msk]).

[jaiabhayk]

@yindaheng98 one QQ:
how this is common after undistortion by colmap ?

[yindaheng98]

@yindaheng98 one QQ: how this is common after undistortion by colmap ?

Oh, sorry for missing the background context.
I'm using Dust3r to reconstruct dynamic scenes from multi-view videos, where multiple cameras simultaneously capture the scene from different viewpoints.
For each frame, I first run COLMAP to estimate camera poses and perform image undistortion, and then use Dust3r for dense reconstruction.
Since each camera typically has slightly different lens distortion parameters, the undistortion step performed by COLMAP often results in images with slightly different resolutions.
This occurs because COLMAP crops the image corners differently for each camera to remove radial and potentially tangential distortions, as explained here: colmap/colmap#1100 (comment).


[1] T. Li et al., “Neural 3D Video Synthesis From Multi-View Video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022, pp. 5521–5531.