scheduler in rdm training

[JustinXu0]

I noticed that when training RDM, we need to set args.cosine_lr=True to initialize the scheduler in engine_rdm.py. However, the instructions given in the readme defaults to args.cosine_lr=False. I am new in deep learning. I wonder is it correct to keep the learning rate of adamw as a constant during training? Why can it still converge? Looking forward to your reply!

[LTH14]

You don't need to set args.cosine_lr=True. A constant learning rate should be ok and can still converge -- the convergence of a model does not mean that the learning rate is 0, but instead, the loss stops decreasing.