Title of Abstract: The Heavy-Tailed Nature of Noise in Training Neural Networks
Name of Mentor: Thomas Pietraho
Mentor’s Organization or Department: Department of Mathematics, Bowdoin College
Research Abstract: Artificial neural networks are machine-learning algorithms used to forecast events and optimize systems in a wide scope of areas including technology, finance, and mechanics. We analyzed the randomness that occurs in the machine-learning optimization process and found that the optimization process is non-Gaussian. Additionally, we found that the larger the neural network, the less Gaussian and more risky the optimization process behaves. Our findings provide insight for how to create better neural networks.