Easy Questions First? A Case Study on Curriculum Learning for Question Answering

By Mrinmaya Sachan, Eric P. Xing. 2016

96 citations recorded by the source corpus.

Cognitive science researchers have emphasized the importance of ordering a complex task into a sequence of easy to hard problems. Such an ordering provides an easier path to learning and increases the speed of acquisition of the task compared to conventional learning. Recent works in machine learning have explored a curriculum learning approach called self-paced learning which orders data samples on the easiness scale so that easy samples can be introduced to the learning algo-rithm ﬁrst and harder samples can be introduced successively. We introduce a number of heuristics that improve upon self-paced learning. Then, we argue that incorporating easy, yet, a diverse set of samples can further improve learning. We compare these curriculum learning proposals in the context of four non-convex models for QA and show that they lead to real improvements in each of them.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

UnifiedQA: Crossing Format Boundaries With a Single QA System (2020, methodology): “A more sophisticated teaching curriculum (Sachan and Xing, 2016) or approaches such as model distillation and teacher annealing (Clark et al.”
Coarse-to-Fine Question Answering for Long Documents (2016, background): “Curriculum learning was investigated in Sachan and Xing (2016), but they focused on the ordering of training examples while we combine supervision signals.”
Self-Training for Jointly Learning to Ask and Answer Questions (2018, methodology): “While curriculum learning has seldom been used in NLP, we draw some ideas for curriculum learning from Sachan and Xing (2016a) who conduct a case study of curricu-”
Learning to Teach (2018, methodology): “We mainly use three parts of such signals in our classiﬁcation tasks: 1) the predicted probabilities of each class; 2) the loss value on that data, which appears frequently in self-paced learning (Kumar et al., 2010; Jiang et al., 2014a; Sachan & Xing, 2016); 3) the margin value.”
A Competence-aware Curriculum for Visual Concepts Learning via Question Answering (2020, methodology): “The competence-aware curriculum in our work is related to curriculum learning [7,53,55,20,51,44,21,47] and machine teaching [61,62,38,11,40,15,58].”
Training Curricula for Open Domain Answer Re-Ranking (2020, background): “timating the quality of weak labels (namely, treating BM25 scores as labels), rather than the difficulty of training samples with higher-quality labels (e.g., human-annotated labels). Sachan and Xing [34] propose curriculum learning approaches for question answering, but in a closed-domain setting. ”
Learning by Asking Questions (2017, methodology): “Inspired by curriculum learning [5, 6, 29, 45], we use this difference to select questions on which the answering module can improve the fastest.”