Google AI Blog: A New Lens on Understanding Generalization in Deep Learning

Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University Understanding generalization is one of the fundamental unsolved pr...

A New Lens on Understanding Generalization in Deep Learning Skip to main content A New Lens on Understanding Generalization in Deep Learning March 10, 2021 Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University Quick links Share Copy link × Understanding generalization is one of the fundamental unsolved problems in deep learning. Why does optimizing a model on a finite set of training data lead to good performance on a held-out test set ? This problem has been studied extensively in machine learning, with a rich history going back more than 50 years. There are now many mathemat

related reading

• Just Ask for Generalization | Eric Jangevjang.com
• Understanding deep learning requires rethinking generalizationarxiv.org
• Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsmathai-iclr.github.io
• The Little Book of Deep Learningfleuret.org
• The Scaling Hypothesis · Gwern.netgwern.net
• [2201.02177] Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsarxiv.org
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• The Decade of Deep Learning | Leo Gaobmk.sh
• [2503.02113] Deep Learning is Not So Mysterious or Differentarxiv.org
• Elon Litman | Elements of a Vector Spaceelonlit.com
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org