[Finetune] print metrics by rank0 for train and evaluation in fine-tuning

[harborn]

print metrics by rank0 for train and evaluation in fine-tuning
example:

 ***** train metrics *****                                                                                                                                                                                                                                     
   epoch                    =        1.0                
   throughput               =     0.0583
   total_flos               =   605304GF
   train_loss               =     0.8543
   train_runtime            = 0:06:52.01
   train_samples_per_second =      0.058
   train_steps_per_second   =      0.005
 ***** eval metrics *****
   epoch                   =        1.0                
   eval_loss               =        nan
   eval_runtime            = 0:01:24.54
   eval_samples_per_second =      0.012
   eval_steps_per_second   =      0.012

here throughput is samples/sec

[carsonwang]

Do you need to multiply the number of epoch? Can you please share the result of one of your runs? How does the result compare with the result on Gaudi dashboard?