A high-bias, low-variance introduction to Machine Learning for physicists

By Pankaj Mehta, Marin Bukov, Ching-Hao Wang, Alexandre G. R. Day, Clint Richardson, Charles K. Fisher, David J. Schwab. Physics reports, 2018

1,003 citations recorded by the source corpus.

Machine Learning (ML) is one of the most exciting and dynamic areas of modern research and application. The purpose of this review is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists. The review begins by covering fundamental concepts in ML and modern statistics such as the bias-variance tradeoff, overfitting, regularization, generalization, and gradient descent before moving on to more advanced topics in both supervised and unsupervised learning. Topics covered in the review include ensemble models, deep learning and neural networks, clustering and data visualization, energy-based models (including MaxEnt models and Restricted Boltzmann Machines), and variational methods. Throughout, we emphasize the many natural connections between ML and statistical physics. A notable aspect of the review is the use of Python Jupyter notebooks to introduce modern ML/statistical packages to readers using physics-inspired datasets (the Ising Model and Monte-Carlo simulations of supersymmetric decays of proton-proton collisions). We conclude with an extended outlook discussing possible uses of machine learning for furthering our understanding of the physical world as well as open problems in ML where physicists may be able to contribute.

View Paper: https://www.semanticscholar.org/paper/36647d4e43c2483f2007149a2a72ec7ef8784840

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe (2022, background): “Machine learning is a field of computer science which has been attracting much attention in many areas in physics [115, 313, 417].”
Machine Learning for High-entropy Alloys: Progress, Challenges and Opportunities (2022, methodology): “For example, when the dataset size is small, a simple linear regression should perform better than a deep neural network model due to the well-known bias-variance tradeoff [87].”
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning (2020, background): “Some prior works also try to attribute the benefit of ensemble as reducing the variance of individual solutions [16, 62, 64, 82, 83] due to label noise or non-convex landscape of the training objective (so some individual models might simply not be trained very well by over-fitting to the label nois”
Machine learning and the physical sciences (2019, methodology): “Experiments presented on the Ising model in (Mehta et al., 2018) provide some preliminary thoughts in that direction.”
DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks (2018, background): “A growing number of physics tasks utilize machine learning techniques (see Mehta et al. (2018) for a recent review).”
Parameterized quantum circuits as machine learning models (2019, background): “…is pedagogical and not overly-technical, although we assume familiarity with basic machine learning deﬁni-tions and methods (see Mehta et al. [13] for a physics-oriented introduction to machine learning), and basic working knowledge on quantum computing (see Nielsen and Chuang [14], Chapter…”
Big-Data Science in Porous Materials: Materials Genomics and Machine Learning (2020, methodology): “r the prediction of gas adsorption properties property labeled RDFs have already been introduced by Fernandez et al.212 The property labeled RDF is given by RDFP = f å i,j P iP j exp h B r ij R 2 i , (7) where P i and P j are elemental properties of atom i and j in a volume of radius R. B is a smoot”