CUDA out of memory issue when using with pretrained weight from COOP #9
Description
Hi, I was reproducing TPT with loading pretrained weight from COOP .
I realized the current code directly loads pretrained weight which is mapped at gpu index:0.
This causes the current code to map at global gpu index 0 which is not what I want.
I think the loading pretrained context should be mapped to 'cpu' and then copy the weight from there.
This will save some memory that was holding up pretrained weight.