PAC Generalization Bounds for Co-training

By Sanjoy Dasgupta, Michael L. Littman, David A. McAllester. Neural Information Processing Systems, 2001

339 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/a48abd949978432b8a9498f94b029e440bc1b4f3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Multi-view Regression Via Canonical Correlation Analysis (2007, methodology): “…no further distributional assumptions (aside from a bounded second moment on the output variable), such as the commonly used, overly-stringent assumption that the distribution of the views be conditionally 3 independent given the label [Blum and Mitchell, 1998, Dasgupta et al., 2001, Abney, 2004].”
Analyzing Co-training Style Algorithms (2007, background): “Dasgupta et al. [7] proved that when the requirement of suﬃcient and redundant views is met, the co-trained classiﬁers could make fewer generalization errors by maximizing their agreement over the unlabeled data.”
Effective Self-Training for Parsing (2006, background): “Gildea (2001) and Bacchiani et al.”
Co-training Improves Prompt-based Learning for Large Language Models (2022, background): “Many subsequent analyses (e.g., Dasgupta et al., 2002; Balcan et al., 2005) relax this condition, showing that views can be dependent or even identical as long as certain relationships hold between the models being trained (essentially, they are “different enough”).”
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss (2021, methodology): “The original co-training algorithms (Blum and Mitchell, 1998, Dasgupta et al., 2002) assume that the two views are independent conditioned on the true label and leverage this independence to obtain accurate pseudolabels for the unlabeled data.”
Inductive multi-task learning with multiple view data (2012, background): “Co-training [6] iteratively labels unlabeled examples using the models built from existing labeled examples, and expands the pool of labeled examples until convergence of performance [14, 20].”
Multi-view clustering (2004, background): “[7] and Abney [1] have made the important observation that the disagreement between two independent hypotheses is an upper bound on the error risk of either hypothesis.”
Cross-Domain Learning from Multiple Sources: A Consensus Regularization Perspective (2010, background): “In addition, Dasgupta et al. [25] and Abney [26], [27] provided PAC bounds on the error of co-Training in terms of the disagreement rate of hypotheses on unlabeled data in two independent views.”
Efficient co-regularised least squares regression (2006, background): “Dasgupta et al. (2001) and Leskes (2005) give bounds on the error of co-training in terms of the disagreement rate of hypotheses on unlabelled examples in two independent views.”
Asymmetric Tri-training for Unsupervised Domain Adaptation (2017, methodology): “The generalization ability of co-training is theoretically ensured (Balcan et al., 2004; Dasgupta et al., 2001) under some assumptions and applied to various tasks (Wan, 2009; Levin et al., 2003).”
Estimating Accuracy from Unlabeled Data (2014, methodology): “In [Dasgupta et al., 2001] the authors PAC-bound the error rates using the pairwise agreement rates only, under the assumption that the functions make independent errors, and [Madani et al., 2004] estimate the average error of two predictors using their disagreements.”
Introduction to Semi-Supervised Learning (2009, background): “Further theoretical analysis of Co-Training can be found in [12, 10, 53].”