Labeling body-parts as different objects ?

[apenemo]

Hi, I would like to label primates' hands as a different class from the whole body. A practical example would be:

Class 1) Animal's whole body
Class 2) Animal's hand only

Can I do this given that the labelling of this two classes will be partially superposed ?

[yujiahu415]

I think the Detector can distinguish whole body VS hand, but I haven't tested this. You probably can do a quick test with annotating 50 images to train a Detector. Let me know how this goes. I'm also curious and I can help with troubleshooting.

[yujiahu415]

Sure! Let me know how it goes. I suggest you to use the whole image / frame to annotate and don't use cropped picture, because your goal is to let LabGym distinguish the whole body VS the hands, right? Ideally the two classes should be in the same image so that the Detector can learn how to distinguish them. By the way, is there an obvious boundary between the whole body and the hands? If so, the detection task will be much easier.

[apenemo]

Hi Yujia !

I was unclear in my latest reply so I will try to explain myself better. I thought that IF somehow using two classes confuses labgym, maybe a better strategy to have a more accurate detection of the hands would be to simply have high-quality pictures of my species' hands and label them as " an individual".

given that I have a lot of labelled pictures of " full bodied " indivuals, I figured maybe adding cropped pictures of the hands as " individuals" could help the detection of primate's hands on full bodied individuals ?

( by full bodied I mean that the whole body of the individual can be seen on screen,)

concerning the boundary between hands and arm, I would say that the palm of the hand is easily distinguishable but that the other face is more likely to be confused with the rest of the arm because of the continuous fur, well, primates do have nails but the last time I worked on primates with LabGym they seemed difficult to detect.

I would be very glad to discuss it privately with you so that I can share more details about my work.

Best wishes :)

[yujiahu415]

I see. Thanks for the clarification! I don't think cropped images would do any help. So just try annotating the "hand" and the "rest of the body" in one image. If the two side of hand looks very different, probably make them into two separate classes, like "hand inner side" and "hand outer side".
And yes, you can reach me at henryhu@umich.edu. I may also have some suggestions on Categorizer training to improve the accuracy in your case.

[dr-moo]

Let me chime in! I have worked with computer vision across all possible species, and you kind of learn from your own mistakes :-)

Detection uses an instance/semantic segmentation approach, and all the backbone solutions (e.g. Res-Net) are absolutely fine with multiple classes. It is important to annotate the WHOLE frame, as Yujia kindly mentioned. Make sure you adjust your ethogram to the annotation method - a clear definition of what is being annotated and IF Roboflow (for example) allows it. Another step is to describe the actual "behaviour/behavioural state" based on how you annotated your desired classes. Hope this helps!

[yujiahu415]

@dr-moo Thank you for your insightful inputs :)