Misalignment between labels and image frames

[zhangy76]

Hi,

Thank you for releasing this great work!

I noticed that during fast motions, the labels (or annotations) fail to align with the images, resulting in misalignment between the ground truth and the actual frames. Some examples are shown below (the middle and right one, particularly the markers on the arms).

I wonder if this is the inherent data collection issue or my visualization issue?

[sha2nkt]

Hi,

Thanks for trying out the dataset!

There is definitely some error. Can I advise you to follow the steps in the README here?

I can get exact alignment on my end following these steps.

[zhangy76]

Thanks for the reply.

Yes, I was using the README code for the visualization. The visualization looks fine for the static poses but not those with fast movements. Just would like to confirm that such misalignment error can exist, and may present excessively.

[sha2nkt]

The errors you show above do not match my experience with the dataset. I suspect there might be an fps mismatch during visualization. Is that possible? Just fyi, the images are captured at 30 fps whereas the mocap is 60 fps.

If you can share your visualization code, probably I might be able to help more.

[zhangy76]

Yes, the FPS is considered.

To debug, I use the exact released demo code and command this time, but mannually set selected_frame = 80 to choose the frame with body movement. I obtain the following, where errors present at the hand.

220923_yogi_body_hands_03596_Tree_Pose_or_Vrksasana_-a_YOGI_Cam_01_0041_markers

[sha2nkt]

The hand markers are often occluded - so some misalignment on the hands is to be expected.

In the image you share above, the body markers are well-aligned - this is consistent with my experience.

Could you share names of sequences where you saw misalignment on the body markers, especially those in the images you shared in your first post? I will take a look on my end then.

[zhangy76]

Thanks! The issue presents as long as there are fast body movements. The examples I shared are from

221004_yogi_nexus_body_hands_03596_Bound_Angle_Pose_or_Baddha_Konasana_-b