How to train the model on multiple GPUs

Hello, when training the model, despite specifying multiple GPUs, it seems that the model is only training on a single GPU. Is there a way to train the model in parallel across multiple GPUs?

I've noticed the "local_rank" parameter, and I'm not sure if modifying this parameter can enable parallel training. Could you provide further guidance on how to use this parameter? Thank you.