How to train NNUE model?

[swim2sun]

Thank you for your work—this is an excellent project! I'm particularly interested in the NNUE version. I noticed that the code loads a pre-trained model. Could you please explain how the model was trained?

[agnelvishal]

Yes. I also would like to know how the .pickle file was generated.
The Stockfish nnue project generates .nnue file and not .pickle file.

[thomasahle]

I'm afraid I don't have the code anymore. But it was trained on cclr data: https://lczero.org/blog/2018/09/a-standard-dataset/
This data is not enhanced with evaluations, so it just uses win/loss/draw labels. Probably using mse-loss like alpha-zero.
I tried a lot of different compact architectures, as you can see in my w&b below. However, sadly I never got it fast enough to be stronger than the original sunfish :(