[Help Wanted] DistributedDataParallel (DDP) instead of DataParallel (DP)

[lofrienger]

Hi, very tidy and efficient base code!
Do you have a plan to implement DistributedDataParallel (DDP) instead of DataParallel (DP) for training with multiple gpus?

[Ashkf]

DP is suitable for simple implementation of single machine with multiple GPUs, DDP is complicated for this case