Rademacher penalties and structural risk minimization

By Vladimir Koltchinskii. IEEE Transactions on Information Theory, 2001

423 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/329b2bc0d9ee04038af0f65a366d389da71e6b9c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning (2016, background): “MTL learning guarantees are centered around the notion of (global) Rademacher complexities; notions that were introduced to the machine learning community by [7, 9, 34, 31, 30], and employed in the context of MTL by [45, 46, 26, 48, 47, 49].”
Positive-Unlabeled Learning with Non-Negative Risk Estimator (2017, methodology): “As a consequence, we have to use a different Rademacher complexity with the absolute value inside the supremum [35, 36], whose contraction makes the coefficients of (15) doubled compared with Lemma 8 of [19]; moreover, we have to assume G is closed under negation to change back to the standard Radem”
Rejoinder: Local Rademacher complexities and oracle inequalities in risk minimization (2006, methodology): “The idea to use Rademacher processes to construct data-dependent complexity penalties in model selection problems of learning theory was suggested independently by Koltchinskii [26] and Bartlett, Boucheron and Lugosi [4].”
Concentration Inequalities (2008, background): “Quantities like Z have been known to measure effectively the complexity of model classes in statistical learning theory, see, for example, Koltchinskii [53], Bartlett, Boucheron, and Lugosi [45], Bartlett and Mendelson [54], Bartlett, Bousquet, and Mendelson [55].”
Theory of classification : a survey of some recent advances (2005, methodology): “It is pointed out in recent results of Bartlett, Mendelson, and Philips [30] and Koltchinskii [125] that sharper and simpler bounds may be obtained by taking Rademacher averages over level sets of the excess risk rather than on L 2 (P ) balls.”
A Hilbert Space Embedding for Distributions (2007, background): “Note that there is a strong connection between Theorem 2 and uniform convergence results commonly used in Statistical Learning Theory [19] [16].”
Empirical margin distributions and bounding the generalization error of combined classifiers (2002, background): “i=1 gi XikF + 2 p 2E jgj p n : (6) Combining the bounds (2), (3), (6), we prove that with probablity at least 1 expf 2"2ng Pff 0g Pn'( ) + 2 p 1 Ekn 1 n X”
Generalizing from Several Related Classification Tasks to a New Unlabeled Sample (2011, neutral): “…decomposition: sup f∈Bk(R) ∣∣∣∣∣∣ 1N N∑ i=1 1 ni ni∑ j=1 `(f(X̃ij), Yij)− E(f,∞) ∣∣∣∣∣∣ ≤ sup f∈Bk(R) ∣∣∣∣∣∣ 1N N∑ i=1 1 ni ni∑ j=1 ( `(f(P̂ (i)X , Xij), Yij)− `(f(P (i) X , Xij), Yij) )∣∣∣∣∣∣ + sup f∈Bk(R) ∣∣∣∣∣∣ 1N N∑ i=1 1 ni ni∑ j=1 `(f(P (i)X , Xij), Yij)− E(f,∞) ∣∣∣∣∣∣ =: (I) + (II).”
GLOBAL UNIFORM RISK BOUNDS FOR WAVELET DECONVOLUTION ESTIMATORS (2011, methodology): “We suggest a new approach to nonparametric confidence bands based on Rademacher symmetrization, in a similar vein as in recent work of Koltchinskii [29].”
Kernel methods in machine learning (2007, neutral): “We refer to the works of Bartlett and Mendelson [12], Jordan et al. [80], Koltchinskii [86], Mendelson [98] and Vapnik [141] for details.”
Does learning require memorization? a short tale about a long tail (2019, methodology): “Fairly tight data-dependent bounds on the generalization error can be obtained via the notion of empirical Rademacher complexity [Kol01; BM02].”
A Two-Stage Weighting Framework for Multi-Source Domain Adaptation (2011, methodology): “Next, we bound the difference between the true joint error function E α,β(h) and its empirical estimate Ê α,β(h) using the weighted Rademacher complexity measure [17, 18] defined as follows:”
Local Rademacher complexities (2005, methodology): “h known for a long time to be related to the expected supremum of the empirical process (thanks to symmetrization inequalities), it was ﬁrst proposed as an eﬀective complexity measure by Koltchinskii [15], Bartlett, Boucheron and Lugosi [1] and Mendelson [25] and then further studied in [3]. Unfortu”
Universum Prescription: Regularization Using Unlabeled Data (2015, methodology): “We use the formulation by Zhang (2013), but anterior results are in Bartlett et al. (2002), Bartlett & Mendelson (2003), Koltchinskii (2001) and Koltchinskii & Panchenko (2000).”
A statistical approach to rule learning (2006, methodology): “, 2002) and Rademacher averages (Bartlett and Mendelson, 2003; Koltchinskii, 2001).”
Domain Generalization by Marginal Transfer Learning (2017, methodology): “In both cases, a standard approach using the Azuma-McDiarmid inequality (McDiarmid, 1989) followed by symmetrization and Rademacher complexity analysis on a kernel space (Koltchinskii, 2001; Bartlett and Mendelson, 2002) can be applied.”