Self-Paced Learning for Latent Variable Models

By M. Pawan Kumar, Ben Packer, Daphne Koller. 2010

1,658 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, methodology): “, 2009) and SelfPaced Learning (Kumar et al., 2010; Jiang et al., 2015), in which the models are trained with harder Algorithm 1 The Clustering-based Progressive Pretraining”
Webly Supervised Learning of Convolutional Networks (2015, methodology): “After seventh layer, another fully connected layer is used to predict class labels.”
Learning to Execute (2014, methodology): “This approach can be implemented with curriculum learning (Bengio et al., 2009; Kumar et al., 2010; Lee & Grauman, 2011), which prescribes gradually increasing the “difﬁculty level” of the examples presented to the LSTM, and is partially motivated by fact that humans and animals learn much faster…”
Practical Bayesian Optimization of Machine Learning Algorithms (2012, neutral): “A popular example task is the binary classification of protein DNA sequences [18, 20, 19].”
Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education (2015, background): “…developments in machine teaching, such as the sequential teaching setting (Bengio et al. 2009; Khan, Zhu, and Mutlu 2011; McCandliss et al. 2002; Kumar, Packer, and Koller 2010; Lee and Grauman 2011; Cakmak and Lopes 2012; Pashler and Mozer 2013; Kobayashi and Shinohara 2009; Balbach and…”
Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction (2017, methodology): “In this paper, we choose the linear regularizer proposed in [11], which linearly discriminates samples with respect to their loss, as: Similarto[11][13],weadoptAlternativeConvexSearch(ACS)[9] to solve Equation (1).”
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples (2017, methodology): “Some of them require extra time to collect necessary statistics such as the gradient magnitude of each sample [12, 1], change the network architecture [15, 44], or involve an annealing schedule like selfpaced learning [25, 32].”
An Empirical Study of Example Forgetting during Deep Neural Network Learning (2018, background): “Techniques of note — among others — are predeﬁned curricula of examples (Bengio & LeCun, 2007), self-paced learning (Kumar et al., 2010), and more recently meta-learning (Fan et al., 2017).”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, methodology): “ the curriculum requires at least some manual speciﬁcation. A key point about our work is that Alice and Bob devise their own curriculum entirely automatically. Previous automatic approaches, such as [9], rely on monitoring training error. But since ours is unsupervised, no training labels are requi”
BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning (2018, background): “There is abundant evidence in prior literature which shows that a curriculum may greatly facilitate learning of complex tasks for neural architectures (Bengio et al., 2009; Kumar et al., 2010; Zaremba and Sutskever, 2015; Graves et al., 2016).”
Machine Teaching for Bayesian Learners in the Exponential Family (2013, background): “The third thread is the research on better ways to training machine learners such as curriculum learning or easy-to-hard ordering of training items [3, 15, 16], and optimal reward design in reinforcement learning [21].”
Reverse Curriculum Generation for Reinforcement Learning (2017, background): “One particular type of curriculum learning explicitly enables the learner to reject examples which it currently considers too hard [12, 13].”
Learning to Reweight Examples for Robust Deep Learning (2018, methodology): “…weights are typically calculated based on the training loss, as in many classical algorithms such as AdaBoost (Freund & Schapire, 1997), hard negative mining (Malisiewicz et al., 2011), self-paced learning (Kumar et al., 2010), and other more recent work (Chang et al., 2017; Jiang et al., 2017).”
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020, background): “, 2009) and self-paced learning (Kumar et al., 2010; Lee and Grauman, 2011) where all examples are prioritized based on their “difficulty”.”
Meta-Transfer Learning Through Hard Tasks (2019, methodology): “[62] introduced an iterative self-paced learning algorithm where each iteration simultaneously selects easy samples and learns a new parameter vector.”
Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels (2018, neutral): “Interestingly, the resulting algorithm is similar to that of self-paced learning [21].”
Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence (2019, methodology): “Inspired by curriculum learning (Ben-gio et al., 2009) and self-paced learning (Kumar et al., 2010) literature, the DOS induces a curriculum for the new class samples in a self-paced manner, thus allowing our model to achieve better intransigence on learning the new classes.”