BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning

By Andreas Kirsch, Joost R. van Amersfoort, Yarin Gal. 2019

777 citations recorded by the source corpus.

We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning. BatchBALD is a greedy linear-time $1 - \frac{1}{e}$-approximate algorithm amenable to dynamic programming and efficient caching. We compare BatchBALD to the commonly used approach for batch data acquisition and find that the current approach acquires similar and redundant points, sometimes performing worse than randomly acquiring data. We finish by showing that, using BatchBALD to consider dependencies within an acquisition batch, we achieve new state of the art performance on standard benchmarks, providing substantial data efficiency improvements in batch acquisition.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Coresets via Bilevel Optimization for Continual Learning and Streaming (2020, methodology): “The results for summarizing MNIST are shown in Figure 3a, where we plot the test accuracy against the summary size over 5 random seeds for uniform sampling, coreset generation, and for the state-of-the-art in active learning for our chosen CNN architecture [33].”
A survey of uncertainty in deep neural networks (2021, background): “Notable examples include Bayesian model selection [109], [110], [111], [112], model compression [76], [113], [114], active learning [115], [23], [116], continual learning [117],”
Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds (2019, background): “There is also a large body of literature on batch mode active learning, where the learner is asked to select a batch of samples within each round (Guo and Schuurmans, 2008; Wang and Ye, 2015; Chen and Krause; Wei et al., 2015; Kirsch et al., 2019).”
A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis (2019, methodology): “While this method has been shown to be particularly e ﬀ ective for AL, when querying batches of data-points, it often results in many very similar, redundant data-points being acquired when used in a greedy fashion, as such BatchBALD was introduced to alleviate this problem Kirsch et al. (2019).”
Directions in Abusive Language Training Data: Garbage In, Garbage Out (2020, background): “The challenges posed by small datasets could potentially be overcome through machine learning techniques such as ‘semi-supervised’ and ‘active’ learning [66], although these have only been limitedly applied to abusive content detection so far [65].”
Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation (2021, background): “, with batch Bayesian optimization (Kirsch et al., 2019).”
A Survey of Deep Active Learning (2020, methodology): “In addition, previous heuristicbased AL [139] query strategies have proven to be ineffective when applied to DL [138]; therefore, for the one-by-one query strategy in classic AL, many researchers focus on the improvement of the batch sample query strategy [10, 51, 84, 183], taking both the amount of”
Directions in abusive language training data, a systematic review: Garbage in, garbage out (2020, background): “The challenges posed by small datasets could potentially be overcome through machine learning techniques such as ‘semi-supervised’ and ‘active’ learning [93], although these have only been limitedly applied to abusive content detection so far [94].”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “[104] proposed BatchBALD, which uses greedy algorithm to select a batch in linear time and reduce the run time.”
Towards Robust and Reproducible Active Learning using Neural Networks (2020, background): “The promise of reducing annotation cost has brought a surge of interest in AL research [2, 3, 7, 14, 16, 25, 27, 28, 32] and with it, a few outstanding issues.”