About training

[chenzean]

I am in the process of training my own dataset and found that after 200000 iterations, the loss does not show a decreasing trend. May I ask if this is normal?

[jiangyuhangcn]

I am in the process of training my own dataset and found that after 200000 iterations, the loss does not show a decreasing trend. May I ask if this is normal?

Hi！I have the same problem as you. Have you solved it? How long have you been training? Has loss converged？ thanks！

[Rakesh-123-cryp]

Change the lr scheduler if you're using, to a less aggressive one. and increase dataset size

[qianduodu]

我正在训练自己的数据集，发现经过 200000 次迭代后，损失并没有呈现下降趋势。请问这是否正常？

为什么我的损失是从十几万开始下降，下降的很慢，而且下降到十几就开始跳动这个原因是什么呢

[Mmmmwei]

I'd like to ask how long it took everyone to train?

[xiaojiebangbang]

Change the lr scheduler if you're using, to a less aggressive one. and increase dataset size

I used the cosine annealing decay strategy and the adamw optimizer, but I couldn't train so many epochs. I only trained 170epochs, but the effect was not very good. The recovery effect of each block was very different, and some blocks had obvious Gaussian noise, so I didn't know how to solve it.