Principles of Risk Minimization for Learning Theory

By Vladimir Naumovich Vapnik. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

Three Approaches for Personalization with Applications to Federated Learning (2020, methodology): “This can be trained using either standard empirical risk minimization (Vapnik, 1992) or other methods such as agnostic risk minimization (Mohri et al., 2019).”
Deep learning in neural networks: An overview (2014, methodology): “, Wang, Z., Eickholt, J., and Cheng, J. (2009). NN con: improved protein contact map prediction using 2D-recursive neural networks.”
Safety Verification of Deep Neural Networks (2016, background): “Traditionally, the correctness of a neural network classiﬁer is expressed in terms of risk [37], deﬁned as the probability of misclassiﬁcation of a given image, weighted with respect to the input distribution µ of images.”
On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products (2016, background): “The first contribution of this work is to critically examine the foundational statist ical machine learning principles of empirical risk minimizatio n and structural risk minimization [5] from the perspective o f safety.”
Invariant Risk Minimization (2019, background): “This is the ubiquitous Empirical Risk Minimization (ERM) principle [50].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “2 M ’s Training To decrease bitsM + bitsH , we add a regularizing term to E, to punish excessive complexity [4, 5, 91, 294, 155, 135, 97, 170, 169, 104, 290, 7, 290, 87, 286, 319, 100, 102].”
Optimal Ordered Problem Solver (2002, neutral): “Other primitives may include various traditional AI path planners (Russell & Norvig, 1994), artificial neural networks (Werbos, 1974; Rumelhart, Hinton, & Williams, 1986; Bishop, 1995) or support vector machines (Vapnik, 1992) for classifying sensory data written into temporary internal storage, as ”
Generalizing from a Few Examples (2019, background): “is usually used as a proxy for R(h), leading to empirical risk minimization [94, 136] (with possibly some regularizers).”
Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations (2016, methodology): “However, this could require more data than HC methods [23].”
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization (2019, methodology): “Here, we show that we can improve performance by directly optimizing for an estimated upper bound on the robust test loss, using ideas from structural risk minimization (Vapnik, 1992).”
Engineering safety in machine learning (2016, background): “The first contribution of this work is to critically examine the foundational statist ical machine learning principles of empirical risk minimizatio n and structural risk minimization [4] from the perspective o f safety.”
Membership Inference Attacks on Machine Learning: A Survey (2021, methodology): “For a given dataset Dtr ain , ERM tries to find the parameters θ∗ that minimize the following objective function: minRDtr ain (θ ) = 1 N N∑ n=1 L(y(n), f (x (n);θ )), (1) where L(·, ·) is a loss function.”
Out-of-Distribution Generalization via Risk Extrapolation (REx) (2020, methodology): “…simply minimizes the average loss across all training examples, regardless of environment: Empirical risk minimization has strong theoretical foundations in the case of i.i.d. data (Vapnik, 1992), and often works well in practice, outperforming more sophisticated methods (Chen et al., 2019).”
Distributionally Robust Neural Networks (2020, methodology): “Here, we show that we can improve performance by directly optimizing for an estimated upper bound on the robust test loss, using ideas from structural risk minimization (Vapnik, 1992).”