Training iteration time

[aoqianz]

I'm wondering what is the normal training time for each iteration on a RTX4090? I'm training 4096 environments in parallel and each iteration takes more than 20 seconds. The paper states the training can reach 2000 iterations in just 1 hour.