An overview of statistical learning theory

By Vladimir Naumovich Vapnik. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

Citations in the corpus

Deep Learning Scaling is Predictable, Empirically (2017, background): “Vapnik and Chervonenkis deﬁned the VC dimension of a model as the cardinality of the largest set of data points that a model can shatter (Vapnik (1998)).”
Robust Large Margin Deep Neural Networks (2016, methodology): “For example, the VC-dimension of DNN with a hard-threshold non-linearity is equal to the number of parameters in the network, which implies that the sample complexity is linear in the number of parameters of the network.”
Federated Optimization: Distributed Machine Learning for On-Device Intelligence (2016, neutral): “An assessment of asymptotic performance of some optimization algorithms as learning algorithms in large-scale learning problems 2 has been introduced in [13].”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, background): “When one can find exact minimum of the empirical risk, everything reduces to balancing approximation–estimation tradeoff that is the object of abundant literature — see for instance [179].”
A Comparison between Regression, Artificial Neural Networks and Support Vector Machines for Predicting Stock Market Index (2015, methodology): “The basic idea of SVM is to map the training data into higher dimensional space using a nonlinear mapping function and then perform linear regression in higher dimensional space in order to separate the data [36].”
Provably safe and robust learning-based model predictive control (2011, unknown): “However, it is known that learning methods that also incorporate global regularization, such as support vector regression [68,72], can outperform strictly local methods [75].”
Support vector machines experts for time series forecasting (2003, background): “SVMs are a very speci3c type of learning algorithms characterized by the capacity control of the decision function, the use of the kernel functions and the sparsity of the solution [6,28,29].”
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks (2019, background): “Traditional learning theory — which restricts capacity (such as VC dimension (Vapnik, 1999)) to achieve good generalization performance — cannot explain why DNNs with large capacity to memorize randomly labeled datasets (Cybenko, 1989) still possess good generalization ability for real datasets…”