Recommended way to process videos

[AmitMY]

Not an actual issue, more of a clarification request.

I'm trying to do hand pose estimation to a video.

Given this video:

This is what OpenPose returns:

It doesn't return the hands because it can't detect them.

Next, I tried to run this hand-tracking code, the simplest way I can:

for frame in frames:
  hands = detector(frame)

As you can see, there are hands flying out, so I added a detector.reset() before invoking it every time:

Which does work better regarding flying hands, but has its own artifacts.

I then tried to perhaps feed the previously detected hands back:

    if len(last_hands) > 0:
        hands = detector(img, last_hands)
    else:
        hands = detector(img)
    last_hands = hands

or to add a detector.reset() every time no hand is detected:

and for completeness sake, here is a synchronized video of everything I tried:

[all3naby]

@AmitMY can you share the above code?

[AmitMY]

https://github.com/AmitMY/pose-docker/tree/master/Google

You don't have to use docker, the code to run once you have this repository installed is:
https://github.com/AmitMY/pose-docker/blob/master/Google/api/pose_video.py

And there is a utility file for joint positions:
https://github.com/AmitMY/pose-docker/blob/master/Google/api/pose_util.py

[wolterlw]

Hi @AmitMY, sorry for the long wait, this project got kinda low on the priority list. Are you still interested in improving the pipeline?
If so - please send me an email (address in profile) and we could collaborate on it. There's plenty of work to be done starting all the way from hand detection, but I'm kind of reluctant to do it on my own.