Can training without image guidance help reducing false alarms as well

[bit-scientist]

As stated

you can choose a cleaner PCT that removes image guidance. The benefit of this approach is that it doesn't require features from a backbone trained on COCO with heatmap supervision. Instead, it directly converts joint coordinates into compositional tokens, making it easier to perform various visualization and analysis tasks. This approach has a slightly reduced performance impact.

The model is performing badly with IR images, detecting inanimate objects such as chairs and long jackets as person.

Would you (@Gengzigang) recommend training without image guidance to alleviate this? In general, what part of the model should be trained with those chair, jacket images as background, SimMIM or Heatmap trained Swin.
AFAIK, both are responsible for extracting image features. I don't have much time to experiment all combinations, so I would love to know your insights. Thank you!