Question about timestep select with win and lose

[TimZhang001]

index = torch.randint( 0, self.num_ddim_timesteps, (x.shape[0],), device=self.device ).long()

maybe the timestep with win and lose should be same.

The curve of my training process is shown below. Why is it that while my lose_diff is increasing, my win_diff is also increasing? In principle, shouldn't lose_diff continuously increase and win_diff continuously decrease

[CIntellifusion]

I think we don't need to require the timestep on winner and lose should be the same. But we could ensure the timestep in the pair to be same.

and what is the w_diff and l_diff here?

According to the train loss and implicit acc, it works well.

[mengsiwei]

@TimZhang001 Hi, thanks for the training records. Could you provide the training config, or just follow the author's config? I tried to reproduce the training process; however, my loss and implicit_acc did not match yours.
Btw, I am also wondering about the code errors like you mentioned in issue#4.
Thanks for your help!