Linear segmentation evaluation

[bigandlittle]

Hello,

Thank you for your work. Amazing results on both classification and dense prediction tasks!

You have opened the evaluation code for the classification task. Are you planning to open your linear segmentation evaluation code on the Pascal Voc dataset so we can easily reproduce your results?

Sincerely,
Shirley

[Adrien987k]

Hi,

I used the mmseg library: https://github.com/open-mmlab/mmsegmentation for all the segmentation results.

The configs and models to copy into mmseg are available in our segmentation folder and should be enough to reproduce the results.

[bigandlittle]

Hi @Adrien987k ,

Thanks for your quick reply! I would use mmseg to evaluate the segmentation task.

I evaluated a ResNet-50 model downloaded from your github on the Pascal VOC object detection task. The results I got are not good (26.5/57.5/20.0 (AP/AP50/AP75). I wonder if you can tell me what is wrong with my test. I repeated the same test on a ResNet-50 model from PixPro, but got better results. My 2 experiments are described below.

(Exp_1) I downloaded the ResNet-50 backbone (alpha=0.75) "resnet50_alpha0.75.pth" from https://github.com/facebookresearch/VICRegL. I evaluated this model on the Pascal Voc object detection task using the attached codes. I first converted resnet50_alpha0.75.pth to resnet50_alpha0.75.pkl using the command "python convert-pretrain-to-detectron2.py resnet50_alpha0.75.pth ./resnet50_alpha_75.pkl". Then I run the command "
python train_net.py --config-file configs/pascal_voc_R_50_C4_24k_moco.yaml --num-gpus 2 MODEL.WEIGHTS ./resnet50_alpha_75.pkl". The detection results are "26.5/57.5/20.0 (AP/AP50/AP75) ", which can be found in "detection\output\log.txt" in the attachment.
(Exp_2) I downloaded the ResNet50 (100 epochs) model from "https://github.com/zdaxie/PixPro", converted the model to pkl by running " python convert_pretrain_to_d2 pixpro_base_r50_100ep_md5_91059202.pth ./output.pkl (this command can be found in "PixPro transfer/detection)" and evaluated the converted model by running "python train_net.py --config-file configs/pascal_voc_R_50_C4_24k_PixPro.yaml --num-gpus 2 MODEL.WEIGHTS ./output.pkl" and got the result "59.2/83.1/67.1 (AP/ AP50)/AP75)".

detection.zip

Thanks!

[Adrien987k]

Hi,

I don't have the config for detectron2 but a very important detail is that you need to adjust the learning rate. VICRegL uses a different loss than the contrastive loss used by MoCo and PixPro and therefore needs different hyper-parameters for the downstream tasks.

Also, given that the difference in performance is so large, are you sure the model is loaded properly ?

[baibizhe]

Hello.Thanks for your wonderful work. Would you mind telling me which part of output I should pass to LinearHead for segmentation? There are five values in output dictionary , "representation" "embedding" "maps_embedding" "logits" and "logits_val".Thanks a lot!