how to reproduce the result for cityscapes

[SHMCU]

Thanks for the valuable contribution!
I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml
The validation mIoU can only reach to ~73.x %.
my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

[houqb]

Thanks for your interests in this paper. I have tried spnet50 on Cityscapes and the mAP score is 79.5 on the validation set (w/o hard example mining). Please check your configuration. BTW, I have updated some code. So, you can reclone it. Kind regards, Qibin (Andrew)

…

On Wed, Jul 8, 2020 at 5:23 AM SHMCU ***@***.***> wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#13>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADNCN42ZBE3RWSPFTT3HOFLR2OG65ANCNFSM4OTU57GQ> .

[SHMCU]

In your configuration, do you set use_apex=True ?

Thanks,
SHMCU

[SHMCU]

Hi Qibin,

Could you share your configuration file for this spnet50 achieving mIoU 79.5 on Cityscapes val set?

Thanks,
Hai

[dongzhang89]

I have the similar question.

[cm0561]

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

[SHMCU]

If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 ***@***.***> wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[cm0561]

Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&gt;; 发送时间:&nbsp;2021年11月15日(星期一) 凌晨3:38 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 ***@***.***&gt; wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[cm0561]

Do u&nbsp; have a wechat,can i have your wechat so we can discuss the model in&nbsp; other&nbsp; problem,my wechat number is lz_609492668

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&gt;; 发送时间:&nbsp;2021年11月15日(星期一) 凌晨3:38 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 ***@***.***&gt; wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[SHMCU]

What is the batchsize if you are training on one GPU. If it is larger than 12, then it should be fine.  On Sunday, November 14, 2021, 05:31:38 PM PST, cm0561 ***@***.***> wrote: Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&gt;; 发送时间:&nbsp;2021年11月15日(星期一) 凌晨3:38 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 ***@***.***&gt; wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[cm0561]

I train with only&nbsp; one gpu&nbsp; and the batchsize can only be 1 , it will be an error of "cuda out of memory" if I set it 2,&nbsp; and I have a another problem even i set the batchsize 1 , when the model procress the eval it&nbsp; still have the error of "cuda out of memory",&nbsp; my gpu is 1080ti which have 11G memory!

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&gt;; 发送时间:&nbsp;2021年11月15日(星期一) 上午10:46 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) What is the batchsize if you are training on one GPU. If it is larger than 12, then it should be fine.&nbsp; On Sunday, November 14, 2021, 05:31:38 PM PST, cm0561 ***@***.***&gt; wrote: Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

------------------&amp;nbsp;原始邮件&amp;nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&amp;gt;; 发送时间:&amp;nbsp;2021年11月15日(星期一) 凌晨3:38 ***@***.***&amp;gt;; ***@***.******@***.***&amp;gt;; 主题:&amp;nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 ***@***.***&amp;gt; wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[SHMCU]

Okay, try to train with batchsize larger than 12. You can either choose to shrink  the width of the layers or train at lower resolution or change to a smaller backbone.11G memory is not enough to play with segmentation models. You can try some other CV tasks but not these dense prediction task.Batch size 1 will definitely not work. Cause it makes the batchnorm layer fails. Think about how to compute variance if there is only one sample.  On Sunday, November 14, 2021, 10:21:02 PM PST, cm0561 ***@***.***> wrote: I train with only&nbsp; one gpu&nbsp; and the batchsize can only be 1 , it will be an error of "cuda out of memory" if I set it 2,&nbsp; and I have a another problem even i set the batchsize 1 , when the model procress the eval it&nbsp; still have the error of "cuda out of memory",&nbsp; my gpu is 1080ti which have 11G memory!

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&gt;; 发送时间:&nbsp;2021年11月15日(星期一) 上午10:46 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) What is the batchsize if you are training on one GPU. If it is larger than 12, then it should be fine.&nbsp; On Sunday, November 14, 2021, 05:31:38 PM PST, cm0561 ***@***.***&gt; wrote: Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

------------------&amp;nbsp;原始邮件&amp;nbsp;------------------ 发件人: "Andrew-Qibin/SPNet" ***@***.***&amp;gt;; 发送时间:&amp;nbsp;2021年11月15日(星期一) 凌晨3:38 ***@***.***&amp;gt;; ***@***.******@***.***&amp;gt;; 主题:&amp;nbsp;Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13) If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 ***@***.***&amp;gt; wrote: Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled. The only thing I changed is the batchsize, i am using 12 sample in a batch. Could you give me some hints? I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.