Broken depths and masks for specific objects in MegaPose-ShapeNet

[jcorsetti]

Hello,

I have been playing with the MegaPose partition that uses the objects from ShapeNet, and I have noticed that the depth renderings and the masks for certain objects are not correct. Here is an example from the image 000087_000019:

Depth:

Mask:

You can observe that at least 4 objects have an extremely noisy depth map, and the same happens with their mask (which I obtained from the original RLE with the script you provided). I suspect that this is due to some objects transparencies that led to incorrect rendering. The problem appears widespread enough, as I have seen an average of 2-3 broken objects for each scene.

Would it be possible for you to investigate on this issue, and maybe provide a list of the objects that present this behaviour so that they could be excluded?

[astrophotoversum]

Hi Jaime,
I wrote a python script the filters the broken depth masks relatively well. For every depth mask it analyses the magnitude spectrum of the FFT and standard deviation. The majority of the broken depth masks should be found with this script. However, in some cases really small broken depth masks or broken sub-regions are not found (I didn't check every image). You can of course adapt the script according to your needs and regenerate the files (It might take some time).

In the following zip-file you can find a python file and some .txt-files. The python file generates a .txt file for every shard. The lines in the shard-.txt represent the scenes and the corresponding broken depth mask indices (if there are any).

find_broken_depth_masks.zip

If you have any questions, please do not hesitate to contact me!
Cheers,
Lukas

**EDIT:
The upper .zip-file only contained .txt-files for the GSO dataset. Here is a .zip-file that has the .txt-files for ShapeNet:
shapenet_1M_corrupted_depths.zip

[thodan]

Thanks @astrophotoversum for preparing the script!

Do you know how many broken depth images there are in the whole MegaPose dataset? Would it be possible to provide a list of the broken images (so everyone doesn't need to run the script)? We could link the list on the BOP website.

[astrophotoversum]

Hi Tomas,you‘re welcome but sorry that it took so long!So the .zip-file already contains txt.-files for every shard each indicating the broken depth masks for every image. But I can also provide a list that only contains the broken shards/images.Cheers,LukasAm 06.07.2024 um 23:10 schrieb Tomas Hodan ***@***.***>: Thanks @astrophotoversum for preparing the script! Do you know how many broken depth images there are in the whole MegaPose dataset? Would it be possible to provide a list of the broken images (so everyone doesn't need to run the script)? We could link the list on the BOP website. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[zuodexin]

What is the difference between shards in MegaPose-GSO-fixed and shards in MegaPose-GSO ?

[astrophotoversum]

Hi zuodexin,
the MegaPose-GSO/ShapeNetCore-fixed data contain images that are corrected for the flipped object masks (see #103 for more information). Furthermore, in the ***-fixed folders you will find subfolders named "GSO_broken_depth_maps" (for GSO) and "shapenet_1M_corrupted_depths" (for ShapeNet). These folders contain txt.-files for every shard. Every file contains several lines where each line represents an image in the shard and the numbers in each line are the indices of the objects with broken depth maps in that image (the issue described above). The indices are found with the script I provided above.
I hope that helps!
Best!

[LoveJu1y]

@astrophotoversum but now the ***-fixed dataset is gone , the url is reset , could you please provide a new key_to_shard.json which fliter the bad masks or depth? and after filter ,how many valid samples left yet?