Probability of error of some adaptive pattern-recognition machines

By H. Scudder. 1965

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Semi-Supervised Sequence Modeling with Cross-View Training (2018, background): “One of the earliest approaches to semi-supervised learning is self-training (Scudder, 1965), which has been successfully applied to NLP tasks such as word-sense disambiguation (Yarowsky, 1995) and parsing (McClosky et al.”
Snorkel: Rapid Training Data Creation with Weak Supervision (2017, background): “Another type of supervision is self-training [3,44] and co-training [6], which involves training a model or pair of models on data that they labeled themselves.”
Self-Training With Noisy Student Improves ImageNet Classification (2019, methodology): “our model using the self-training framework [66] which has three main steps: 1) train a teacher model on labeled images, 2) use the teacher to generate pseudo labels on unlabeled images, and 3) train a student model on the combination of labeled images and pseudo labeled images.”
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence (2020, background): “This idea was introduced decades ago [27, 40].”
Revisiting Self-Training for Neural Sequence Generation (2019, methodology): “In this work, we revisit a much older and simpler semi-supervised method, self-training (ST, Scudder (1965)), where a base model trained with labeled data acts as a “teacher” to label the unannotated data, which is then used to augment the original small training set.”
Self-training Improves Pre-training for Natural Language Understanding (2020, methodology): “Self-training is a semi-supervised method which uses a teacher model, trained using labeled data, to create synthetic labels for unlabeled examples (Scudder, 1965; Yarowsky, 1995).”
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, background): “learning [38]–[40] for ASR in the context of deep-learning.”
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, background): “Self-training (Scudder, 1965) is a classical semisupervised learning approach, where unlabelled data is equipped with pseudo-labels from a super-”
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, methodology): “The ﬁrst is self-training, where a teacher model is used to improve the performance of a student model by generating labels for the unlabeled set the student can train on [22–24].”
Self-Training for End-to-End Speech Recognition (2019, background): “One such approach, self-training [2], uses noisy labels generated from a model trained on a much smaller labelled data set.”
Semi-supervised learning, causality, and the conditional cluster assumption (2019, methodology): “The second algorithm we propose is loosely related to the ideas of label propagation and self-learning [9].”
w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training (2021, background): “The ﬁrst line of work is self-training [1, 2, 3], also known as pseudo-labeling, where the system starts with training a teacher model using initially available labeled data.”
Rethinking Pre-training and Self-training (2020, methodology): “Our work argues for the scalability and generality of self-training (e.g., [6–8]).”
Speaker-Follower Models for Vision-and-Language Navigation (2018, methodology): “The semi-supervised approach we use is related to model bootstrapping techniques such as self-training [42, 31] and co-training [6] at a high-level.”
Meta Pseudo Labels (2020, methodology): “The methods of Pseudo Labels or self-training [57, 81, 55, 36] have been applied successfully to improve state-ofthe-art models in many computer vision tasks such as image classification (e.”