No importance sampling weights?

[warmpu]

Hello, when implementing the loss function for the policy network, I observed that there were no importance sampling weights and no clipping of the probability ratio. Why is that?

[NTT123]

Hi, the first PPO update has an importance sampling weight of 1 (the policy that generates the data is the same as the one that we are optimizing). Because we are only doing one update step, we can skip the weighting part completely.

[warmpu]

Thank you for your answer！