Hyperparameters for Roberta

[AJSVB]

Dear Authors,

The current code as used in the example notebook gives very good results with Bert, but poor results with Roberta. This is surprising because the papers show that using Roberta allows much better results.
My guess is therefore that some Hyperparameters are different when Roberta was used in the papers. If this is the case, could a good set of hyperparameters for Roberta be disclosed? If this is not the case, how do you explain the drop in performance?

Best Regards,
Antoine