Research LSGO (Large-scale global optimization)

[eugenioLR]

Research LSGO techniques, as they involve optimizing functions with a large number of parameters (exactly what we need for deep learning), maybe this would allow our methods to be applied to larger methods than small MLPs, such as CNN and RNNs (even Transformers, but that might be a stretch...)

[eugenioLR]

Chapter 7 in Numerical optimization by Nocedal J. and Wright S. J.

https://www.sciencedirect.com/science/article/pii/S2210650223002389

Not LSGO, but interesting: https://link.springer.com/chapter/10.1007/3-540-44745-8_27