VC dimension of neural networks

By Eduardo Sontag. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

On the Expressive Power of Deep Neural Networks (2016, background): “Proceedings of the 34 th International Conference on Machine Learning, Sydney, Australia, PMLR 70, 2017.”
Adaptive computation and machine learning (1998, background): “Sontag and is taken from Sontag (1998). At first let us introduce the function : × → defined by”
On the Complexity of Neural Network Classifiers: A Comparison Between Shallow and Deep Architectures (2014, unknown): “Current available results show that the VC-dim is O ( p 2 ) , for networks with arctan ( · ) or tanh ( · ) activation, where p is the number of parameters [45]–[47].”
Predicting diabetes-related hospitalizations based on electronic health records (2018, unknown): “The proof is based on Lemma 2 of [44].”
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review (2017, methodology): “Specifically, they used the theory of VapnikChervonenkis (VC) dimension (see Vapnik & Chervonenkis, 1971, for details and Sontag, 1998, for the first application to ANNs) to demonstrate that hand-crafted feature extraction creates low-dimensional representations.”
A New Look at an Old Problem: A Universal Learning Approach to Linear Regression (2019, background): “For DNNs, the VC dimension is linear with the number of parameters [11], yet the empirical evidence demonstrates that DNNs have state of the art generalization performance.”
The use of backpropagating artificial neural networks in land cover classification (2003, background): “Hush and Horne (1993) deﬁne the VC dimension as a measure of the capability of the system, while Sontag (1998) describes it as a quantity that characterizes the difﬁculty of distribution-independent learning.”
A joint sparse clustering and classification approach with applications to hospitalization prediction (2016, background): “Proof: The proof is based on Lemma 2 of [7].”
The power of quantum neural networks (2020, methodology): “Further, using the VC dimension to bound generalisation error requires several unrealistic assumptions, including that the model has access to infinite data [15,16].”
Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks (2020, background): “Various techniques have been developed to quantify the gap between the expected risk and the empirical risk, such as PAC learning (Hanneke 2016) , VC dimension (Sontag 1998) and Rademacher complexity (Koltchinskii, Panchenko, and others 2002).”