Upscale embeddings for the RADIO Model

[ivanlyubimtsev]

Thank you for your excellent work.

Tell me, is there any way to upscale the embeddings of the RADIO v3 model?/2.5 up to the original resolution of the images?

[gheinrich]

Hello, you might be interested in training your own FeatSharp in order to upsample RADIO features.

[mranzinger]

FeatSharp works best between 2x and 4x upsampling, so if you're looking for 16x upsampling, you have a few options:

1. Run RADIO itself at 16x the resolution you want
2. Use a different upsampler (I recommend JAFAR)

Note: It's not that FeatSharp can't support 16x, but the way I have it implemented, you'll probably crash with an OOM during training.

Note 2: JAFAR is a great upsampler, however it operates on the low-res feature map, which means that it won't be able to introduce features for now-visible small objects. This can be important as JAFAR is unlikely to improve AP-small object detection, for example, and this is also where FeatSharp shines, as the tiling recovers a lot of small object detail.