Neural Network Learning: Theoretical Foundations

By Martin H. Anthony, Peter L. Bartlett. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

Citations in the corpus

On the Number of Linear Regions of Deep Neural Networks (2014, background): “There is a wealth of literature discussing approximation, estimation, and complexity of artiﬁcial neural networks (see, e.g., Anthony and Bartlett 1999).”
Towards the Science of Security and Privacy in Machine Learning (2016, background): “Such an interpretation helps discover, from speciﬁc attacks and defenses, general principles of adversarial learning that apply across all supervised ML. PAC model of learning: PAC learning model has a very rich and extensive body of work [22].”
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian (2019, background): “Statistical learning theory: Statistical properties of neural networks have been studied since 1990’s [3, 8, 7].”
A theory of learning from different domains (2010, methodology): “The uniform convergence bound on the α -error is nearly identical to the standard uniform convergence bound for hypothesis classes of ﬁnite VC dimension (Vapnik 1998; Anthony and Bartlett 1999), only with target and source errors weighted differently.”
Fair Regression: Quantitative Definitions and Reduction-based Algorithms (2019, methodology): “However, if F has a bounded pseudo-dimension, which always equals the VC dimension of H (Anthony & Bartlett, 2009), then the pseudo-dimension can be used to bound R n ( H ) (see Theorem 6 of Bartlett & Mendelson, 2002).”
Deep Neural Networks for Estimation and Inference: Application to Causal Effects and Other Semiparametric Estimands (2018, methodology): “In that time, shallow neural networks with smooth activation functions were shown to have many good theoretical properties (White (1992), Anthony and Bartlett (1999), Chen and White (1999)).”
On the optimal estimation of probability measures in weak and strong topologies (2013, background): “Therefore, by [1, Theorem 11.12], we obtain fat ǫ ( G ) ≤ 1 + ⌊ ǫ − 1 ⌋ and [25, Theorem 3.2] ensures that there exists a universal constant c ′ such that for every empirical measure P n , and every ǫ > 0, thereby yielding the result.”
Rademacher and Gaussian Complexities: Risk Bounds and Structural Results (2003, methodology): “Neural network methods (see, for example, Anthony and Bartlett, 1999) use repeated compositions of linear functions with scalar nonlinearities, σ : R → [−1, 1], where σ is typically monotonic and smooth.”
A Model of Inductive Bias Learning (2000, methodology): “This proof follows a similar argument to the one presented in Anthony and Bartlett (1999) for ordinary Boolean function learning.”
Robustness and Regularization of Support Vector Machines (2008, background): “Moreover, there has not been an explicit connection to the regularized classi-ﬁer, although at a high-level it is known that regularization and robust optimization are related (e.g., El Ghaoui and Lebret, 1997; Anthony and Bartlett, 1999).”
Exploring Generalization in Deep Learning (2017, background): “Even without making an assumption on the precision of parameters, the VC dimension of feedforward networks can be bounded in terms of the number of parameters dim(w)[1, 3, 6, 23].”
Spectrally-normalized margin bounds for neural networks (2017, methodology): “There are numerous generalization bounds for neural networks, including VC-dimension and fat-shattering bounds (many of these can be found in (Anthony and Bartlett, 1999)).”
On the Computational Efficiency of Training Neural Networks (2014, background): “The sample complexity of such neural networks is well understood [3].”
Stronger generalization bounds for deep nets via a compression approach (2018, background): “, covering number arguments Dudley [2010], Anthony and Bartlett [2009]. There, one needs to fix a hypothesis class independent of the training set.”
A learning theory approach to non-interactive database privacy (2008, background): “Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.”
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks (2018, background): “Besides classical VC-dimension based results (Vapnik, 2013; Anthony and Bartlett, 2009), a vast literature have recently studied the connection between the generalization performance of deep neural networks and their architectures (Neyshabur et al., 2015, 2017a,b; Bartlett et al., 2017; Golowich et…”