Learning how to Active Learn: A Deep Reinforcement Learning Approach
By Meng Fang, Yuan Li, Trevor Cohn. Conference on Empirical Methods in Natural Language Processing, 2017
313 citations recorded by the source corpus.
Active learning aims to select a small subset of data for annotation such that a classifier learned on the data is highly accurate. This is usually done using heuristic selection methods, however the effectiveness of such methods is limited and moreover, the performance of heuristics varies between datasets. To address these shortcomings, we introduce a novel formulation by reframing the active learning as a reinforcement learning problem and explicitly learning a data selection policy, where the policy takes the role of the active learning heuristic. Importantly, our method allows the selection policy learned using simulation to one language to be transferred to other languages. We demonstrate our method using cross-lingual named entity recognition, observing uniform improvements over traditional active learning algorithms.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/d0edf834c1d86aac1835e23c11b23b8723b65136
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- An Overview of Natural Language State Representation for Reinforcement Learning (2020, methodology): “CNNs representations combined with predictive marginals were used in active learning for NER (Fang et al., 2017).”
- Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “Recently, there are some work in these directions [165].”
- Learning to Actively Learn Neural Machine Translation (2018, background): “A crucial difference of our setting to the previous work (Fang et al., 2017; Liu et al., 2018) is that the AL agent receives the reward from the oracle only after taking a sequence of actions, i.e. selection of an AL batch which may correspond to multiple training minibatches for the underlying NMT…”
- SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup (2020, background): “…uncertainty-based (Scheffer et al., 2001; Culotta and McCallum, 2005; Kim et al., 2006) and committee-based approaches (Dagan and Engelson, 1995) 2017) further improve the aforementioned active learning approaches to improve the sampling diversity as well as the model’s generalization ability…”
- Semi-Automated Data Labeling (2020, methodology): “More advanced approaches use reinforcement learning Fang et al. (2017) and deep learning Liu et al. (2018) to train custom selection models.”
- Learning to Teach in Cooperative Multiagent Reinforcement Learning (2018, background): “Several works meta-learn active learning policies for supervised learning (Bachman, Sordoni, and Trischler 2017; Fang, Li, and Cohn 2017; Pang, Dong, and Hospedales 2018; Fan et al. 2018).”
- Massively Multilingual Transfer for NER (2019, background): “The resulting cross-lingual embeddings are then used in NER direct transfer in a leave-one-out setting for the 41 languages ( 41 × 40 transfers), and we report the mean F 1 in Table 3.”
- Zero-shot Text Classification via Reinforced Self-training (2020, background): “There have been some works applying reinforcement learning to data selection in semi-supervised learning, including active learning (Fang et al., 2017), self-training (Chen et al., 2018), co-training (Wu et al., 2018).”
- A Survey of Deep Active Learning (2020, unknown): “ a new possible direction for sample querying. 3.1.5 Other methods. There are also some existing studies that are not as focused as the above query methods; we will summarize them below. For example, [37] redefines the heuristic AL algorithm as a reinforcement learning problem and introduces a new d”