Learning Active Learning from Data

By Ksenia Konyushkova, Raphael Sznitman, Pascal Fua. Neural Information Processing Systems, 2017

346 citations recorded by the source corpus.

In this paper, we suggest a novel data-driven approach to active learning (AL). The key idea is to train a regressor that predicts the expected error reduction for a candidate sample in a particular learning state. By formulating the query selection procedure as a regression problem we are not restricted to working with existing AL heuristics; instead, we learn strategies based on experience from previous AL outcomes. We show that a strategy can be learnt either from simple synthetic 2D datasets or from a subset of domain-specific data. Our method yields strategies that work well on real data from a wide range of domains.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/81ea29bde0216e41420c4591bebb800142fa3269

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Towards more Reliable Transfer Learning (2018, background): “carcity dierently than transfer learning by querying new labels from an oracle for the most informative unlabeled data. A fruitful line of prior work has developed various active learning strategies [20,14,18]. Recently, the idea of combining transfer learning and active learning has attracted incre”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “Whereas “learning active learning from data” (Konyushkova et al., 2017) provides a recent eﬀort to train a meta-learning based regressor to predict expected error reduction for binary classiﬁcation using random forests, the idea is yet to be broadly adapted to deep neural networks.”
What Do I Annotate Next? An Empirical Study of Active Learning for Action Localization (2018, methodology): “To generate a matrix F that explores enough pairs of model and sample states, we follow the Monte-Carlo procedure used in [40].”
Meta-Learning in Neural Networks: A Survey (2020, background): “query policy, (iv) performing outer-level optimization to train the optimal annotation query policy [187], [188], [189].”
A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis (2019, background): “Konyushkova et al. (2017) propose ’Learning Active Learning’, where a regression model learns data selection strategies based on experience from previous AL outcomes.”