Bayes_extensive

Code for the paper : Bayes-optimal learning of random extensive-width networks (link to paper)

Bayes-optimal generalization errors

Multilayer_BO.ipynb implements the theoretical characterization of equations (12)-(14) for the Bayes-optimal regression error $\epsilon_{g,reg}^{BO}$ (4) associated to the learning of the target (2).
Multilayer_BO_classification.ipynb implements the theoretical characterization of equation (15) for the Bayes-optimal classification error $\epsilon_{g,reg}^{BO}$ (5) associated to the learning of the target (2).

(Linear regression) Multilayer_Ridge_repl.ipynb returns the test error achieved by ridge regression, as characterized in equations (19) and (20).
(Logistic regression) Multilayer_Logistic_repl.ipynb returns the test error achieved by logistic regression, discussed in subsection 4.4, see equation (275) in Appendix H.
(Ridge classification) Multilayer_l2_class_repl.ipynb returns the test error achieved by ridge classification, discussed in subsection 4.4, see equation (272) in Appendix H.
(Random features) Multilayer_RF_repl.ipynb returns the test error achieved by ridge classification characterized in (23). The corresponding infinite-width (kernel) limit is given in Multilayer_kernel_lim_repl.ipynb, see equation (24).

Versions: These notebooks employ Python 3.12 , and Pytorch 2.5.

Name		Name	Last commit message	Last commit date
Latest commit History 7 Commits
figures		figures
Multilayer_BO.ipynb		Multilayer_BO.ipynb
Multilayer_Classification_BO.ipynb		Multilayer_Classification_BO.ipynb
Multilayer_Kernel_lim_repl.ipynb		Multilayer_Kernel_lim_repl.ipynb
Multilayer_Logistic_ERM.ipynb		Multilayer_Logistic_ERM.ipynb
Multilayer_RF_repl.ipynb		Multilayer_RF_repl.ipynb
Multilayer_Ridge_repl.ipynb		Multilayer_Ridge_repl.ipynb
Multilayer_l2_class_ERM.ipynb		Multilayer_l2_class_ERM.ipynb
README.md		README.md