Transform between depth and rgb cameras

[abhaybd]

I'm trying to get the RGB point cloud for the individual scans, including the environment besides the object. This should be possible given the depth and rgb images, but the relevant camera matrices and transforms aren't specified.

I see that a camerainfo.npy file is provided, which I assume is the camera matrix for the RGB camera, but this isn't enough since it's also necessary to know the depth camera matrix and relative transform in order to transform the depth cloud into the RGB camera frame.

I already checked if the depth images were already transformed into the RGB frame, but they are misaligned with the color images, so that clearly isn't the case.

Can you provide the camera matrix and relative transform for the depth camera?

[abhaybd]

After some more inspection, it seems the depth image at least has the same intrinsics as the color image, (i.e. specified by camerainfo.npy), but for some reason the point cloud is just misaligned by a fixed amount.

Empirically I found that moving the depth point cloud by 0.021 in the +x direction successfully aligns the depth and color images. I'm not sure why this is the case, and calibrated parameters would definitely be preferable to this hack.

This is the RGB point cloud without the shift:

After applying the shift, the alignment is much better.