Train a regression model

[hu2274898]

Hi, thank you for your excellent work. I would like to use your encoder to train a regression model as a downstream task, but I noticed that your code only supports segmentation and classification. Do you have any suggestions or guidelines on how to train a regression task?

[hu2274898]

I looked into mmpretrain and found many models (such as Vision Transformer) on their GitHub. Can I directly apply the pre-trained weights of UltraSam to a Vision Transformer and then train it for downstream tasks? Your code is used for segmentation and classification, and I’m not sure how to modify the code to adapt it for regression tasks.

[Chop1]

Hi,
I would reuse the UltraSam backbone by replacing the backbone of the method you’re using. If you use MMPretrain, update the backbone section in the config and load the pretrained weights, while keeping the rest of the regression model unchanged.

model = dict(
....
    backbone=dict(  # this modify the parent config backbone
        # _delete_=True,   # use this is you inherit from parent config to erase the parent backbone
        type="mmpretrain.ViTSAM",
        arch="base",
        img_size=1024,
        patch_size=16,
        out_channels=256,
        use_abs_pos=True,
        use_rel_pos=True,
        window_size=14,
        drop_path_rate=0.6,
          init_cfg=dict(
              type="Pretrained",
              prefix="backbone.",
              checkpoint="your_path_to_ultrasam_ckpt.pth",
          ),
    ),
....

Hope this help

[hu2274898]

Thank you for your answer. Do you mean that I should build the framework of my own regression model and then replace the backbone with ViT-SAM? I noticed that MMPretrain only supports classification tasks, so would it be difficult to directly modify your code?

[Chop1]

Hi, I think there is many way to go, i would simply implement my own regression head has an mmpretrain module and plug it in the config file.