Can nnfusion train VHR images whoes feature map in a CNN model exceeds the gpu memory?

[AugustineYang]

Hi. I'm trying to do some segmentation tasks on FCN32s on some 25000*25000 RGB RS images. In your paper of "Welder: Scheduling Deep Learning Memory Access via Tile-graph", you mentioned that your work supports handling DNN models with large input (e.g. high-resolution images). But I still get "CUDA out of memory" ERROR following the instruction How to use NNFusion Python interface for inference/training here. I guess it comes from "PTTrainer(model, loss_func, "cuda:0")". How should I set the args of the trainer to achieve the goal? Thanks for your help.

[AugustineYang]

It seems that "Welder" is implemented in a seperated branch other than "main" branch. What should I do to make Welder engage in the compilation stage?

[LeiWang1999]

checkout this branch https://github.com/microsoft/nnfusion/tree/osdi2023welder or https://github.com/nox-410/Welder

[AugustineYang]

checkout this branch https://github.com/microsoft/nnfusion/tree/osdi2023welder or https://github.com/nox-410/Welder

Thanks for your reply. Does Welder provide python interface for training or inference, just like this link How to use NNFusion Python interface for inference/training?