The alignment loss converges very fast

[Lele-Xie]

Hi, I have tried reimplementing the alignment loss recently. The alignment loss seemed to converge very fast. After 200 steps, the alignment loss decreased to 0.02, while the cross entrope loss was 1.0 around. Is that a normal phenomenon?
Look forward to your kind reply, thank you~

[tuzhaopeng]

Hi Lele, could you specify which project you are referring to — RLVER, RLVMR, or VISTA?