Problems in perspective image generation

[KangLiao929]

Hi,

Thanks for your great work. I have tried to generate the perspective image from my collected panoramic dataset using python -m siclib.datasets.create_dataset_from_pano --config-name my_pano. However, some rendered results suffer from severe artifacts, especially in the object edges (please refer to the following figures). Could you please elaborate on any reasons for that? How can I eliminate these artifacts?

[veichta]

Thank you very much for pointing this out! I've changed the interpolation mode in the camera which should solve this problem.

[sarlinpe]

It might be ok to use bicubic as long as we clamp the resulting image ([0,1] for float, [0,255] for uint8) since bicubic interp can overshoot.

[veichta]

Good point! I'll update it

[KangLiao929]

Thanks for your reply. I have also fixed this issue by simply setting resize_factor=None for cam.get_img_from_pano function. May I ask why you chose to randomly resize the panoramic image before the perspective sampling?

[veichta]

The motivation behind it is that the resampling of the panorama image can introduce artifacts which the network could use to cheat. That is why we rescale the panorama before sampling to keep these artifacts at a minimum.

[KangLiao929]

Oh, interesting! I have a similar observation that the blur artifacts of the synthetic image would make the neural network easy to find a shortcut to estimate the camera parameters, especially the intrinsic parameters like focal length and radial distortion coefficient. More blur artifacts in the image, larger focal length or distortion involved during dataset construction.

But the previous resize operation of the panorama would introduce edge-like artifacts in the image shown above. Could it be another shortcut for the neural network to estimate the extrinsic parameters? Because these tilted edge artifacts may provide an explicit "spatial clue" for the network to reason the horizontal/vertical orientation in the captured scene.