How was the validation split created?

[akash-kamalesh]

Hey there, was going through the paper and the documentation on the repository and I noticed for the openwebtext dataset, you have used a validation set on which the validation loss was plotted. I wanted to know how the validation split was created from the openwebtext dataset, and what methodology did you use to split the dataset into validation and training.