Analyzing the effectiveness and applicability of co-training

By Kamal Nigam, Rayid Ghani. 2000

1,202 citations recorded by the source corpus.

Recently there has been signi(cid:12)cant interest in supervised learning algorithms that combine labeled and unlabeled data for text learning tasks. The co-training setting [1] applies to datasets that have a natural separation of their features into two disjoint sets. We demonstrate that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not. When a natural split does not exist, co-training algorithms that manufacture a feature split may out-perform algorithms not using a split. These results help explain why co-training algorithms are both discriminative in nature and robust to the assumptions of their embedded classi(cid:12)ers

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Training Deep Neural Networks on Noisy Labels with Bootstrapping (2014, methodology): “Co-training (Blum & Mitchell, 1998; Nigam et al., 2000; Nigam & Ghani, 2000) was similarly-motivated but used a pair of classiﬁers with separate views of the data to iteratively learn and generate additional training labels.”
Unsupervised Deep Embedding for Clustering Analysis (2015, methodology): “Our training strategy can be seen as a form of self-training (Nigam & Ghani, 2000).”
A Survey on Multi-view Learning (2013, methodology): “Blum and Mitchell (1998); Nigam and Ghani (2000); Brefeld and Scheffer (2004); Sindhwani et al. (2005); Yu et al. (2011) and Zhu et al. (2012) used the WebKB data as one of the evaluation datasets.”
Semi-Supervised Self-Training of Object Detection Models (2005, methodology): “The alternative that we chose to evaluate in this work is often called self-training or incremental training [13].”
Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning (2017, background): “Co-training [6, 34] uses a pair of classifiers with disjoint views of data to iteratively learn and generate training labels.”
Semi-supervised Deep Learning with Memory (2018, methodology): “In the SSL literature, the most straightforward SSL algorithm is self-training where the target model is incrementally trained by additional self-labelled data given by the model’s own predictions with high confidence [21, 2, 25].”
Classification in the Presence of Label Noise: A Survey (2014, background): “See [310]–[314] for examples of studies on the effectiveness of co-training.”
Semi-supervised Gender Classification with Joint Textual and Social Modeling (2016, methodology): “We implement the co-training algorithm by randomly splitting the feature space into two disjoint feature subspaces as two views (Nigam and Ghani, 2000).”
The impact of social and conventional media on firm equity value: A sentiment analysis approach (2013, methodology): “, based on the open-source Natural Language Toolkit (NLTK) [11,21,28,32].”
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning (2020, methodology): “nnections to these methods and other types of methods in the following subsections. 3.1. Entropy Regularization Pseudo-labeling has been previously analyzed as an equivalent to Entropy Regularization [34, 27]. The conditional entropy of the class probabilities can be used for a measure of class over”
Adaptive computation and machine learning (1998, methodology): “This approach and several variants have been used for a variety of learning problems, including named entity classification (Collins and Singer, 1999), text classification (Nigam and Ghani, 2000; Ghani, 2001), natural language processing (Pierce and Cardie, 2001), large-scale document classification”
Inductive multi-task learning with multiple view data (2012, methodology): “Co-training [6] iteratively labels unlabeled examples using the models built from existing labeled examples, and expands the pool of labeled examples until convergence of performance [14, 20].”
Weakly-Supervised Neural Text Classification (2018, background): “Self-training [19, 21] is a common strategy used in classic semi-supervised learning scenarios.”
Distributed Distillation for On-Device Learning (2020, methodology): “This idea has some resemblance to the bootstrap methods of co-training and co-Expectation-Maximization (co-EM) where two (Naive Bayes or SVMs) classifiers train on disjoint sets of labeled data, provide (noisy) labels on unlabeled data, and then train on the expanded labeled dataset [34, 35].”
Urban Computing: Concepts, Methodologies, and Applications (2014, result): “Cotraining can generate a better inference result as one of the classi.ers correctly labels data that the other classi.er previously misclassi.ed [Nigam and Ghani 2000].”