ournal of Statistical Mechanics : J Theory and Experiment

By Jeffrey Pennington, Pratik Worah. 2019

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation (2019, background): “This would require taking a joint limit in both the skeleton size, over the data distribution, and over the dimensions {nl}l; see Pennington & Worah (2017; 2018) for analogous settings for 2 or 3 layer networks and Gaussian data.”
On the Convergence Rate of Training Recurrent Neural Networks (2018, methodology): “There is huge literature on using the mean-field theory to study neural networks [11, 15, 36, 41, 45–47, 49, 54, 66–68].”
Surprises in High-Dimensional Ridgeless Least Squares Interpolation (2019, unknown): “This confirms a result previously obtained by Pennington and Worah (2017). Corollary 5.”
A Fine-Grained Spectral Perspective on Neural Networks (2019, background): “[42] Jeffrey Pennington and Pratik Worah.”
Exactly Computing the Local Lipschitz Constant of ReLU Networks (2020, background): “Random networks and Lipschitz Constants: Studying various properties of random neural networks has been a fruitful area of research, see e.g.( Hanin & Rolnick, 2019; Pennington & Worah, 2017) and references therein.”
Generalization of Two-layer Neural Networks: An Asymptotic Viewpoint (2020, methodology): “In addition, similar tools have been used to study two-layer neural networks (Louart et al., 2018; Pennington and Worah, 2017) and related quantities such as the Fisher matrix (Karakida et al., 2018; Pennington and Worah, 2018).”
Asymptotics of Wide Networks from Feynman Diagrams (2019, methodology): “For a previous use of diagrams in this context, see [25].”
KAMA-NNs: Low-dimensional Rotation Based Neural Networks (2019, background): “Several recent results [Pennington and Worah, 2017], [Pennington et al., 2017], [Pennington et al., 2018] focus on understanding statistical properties of unstructured random NNs and their connection to random matrix theory.”
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning (2018, background): “different Heavy-Tailed Universality classes [46, 32, 18, 25, 20, 5, 111, 7, 38, 17, 90, 8, 99]; other work on RMT approaches [133, 120, 114, 112, 87, 87, 137, 85, 126]; other work on statistical physics approaches [132, 52, 117, 125, 137, 119, 113]; work on fitting to noisy versus reliable signal [1”
Capturing the learning curves of generic features maps for realistic data sets with a teacher-student model (2021, background): “The asymptotic behavior of kernel matrices has indeed been the subject of intense scrutiny [45, 62, 46, 11, 63, 48].”
Universal statistics of Fisher information in deep neural networks: mean field approach (2018, methodology): “One widely used approach for this purpose is to consider deep networks with random connectivity and a large width limit [2–14].”