Bayesian Active Learning for Classification and Preference Learning
By Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, Máté Lengyel. 2011
1,125 citations recorded by the source corpus.
Information theoretic active learning has been widely studied for probabilistic models. For simple regression an optimal myopic policy is easily tractable. However, for other tasks and with more complex models, such as classication with nonparametric models, the optimal solution is harder to compute. Current approaches make approximations to achieve tractability. We propose an approach that expresses information gain in terms of predictive entropies, and apply this method to the Gaussian Process Classier (GPC). Our approach makes minimal approximations to the full information theoretic objective. Our experimental performance compares favourably to many popular active learning algorithms, and has equal or lower computational complexity. We compare well to decision theoretic approaches also, which are privy to more information and require much more computational time. Secondly, by developing further a reformulation of binary preference learning to a classication problem, we extend our algorithm to Gaussian Process preference learning.
Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Mohit Bansal
- Jianfeng Gao
- Ruslan Salakhutdinov
- Quoc V. Le
- Trevor Darrell
- Percy Liang
In Probabilistic & Bayesian Methods
- Yarin Gal
- Zoubin Ghahramani
- Richard E. Turner
- José Miguel Hernández-Lobato
- Max Welling
- Yee Whye Teh
- David M. Blei
- Thang D. Bui
In Meta & Continual Learning
- Yarin Gal
- Yee Whye Teh
- Richard E. Turner
- José Miguel Hernández-Lobato
- Richard S. Zemel
- Dustin Tran
- Michael A. Osborne
- Abhinav Kumar Gupta
Citations in the corpus
- Learning Algorithms for Active Learning (2017, methodology): “Houlsby et al. (2011) presented an approach based on Bayesian information theory, in which examples are selected in order to maximally reduce the entropy of the posterior distribution over classifier parameters.”
- Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples (2017, methodology): “Notice that there are other methods that can measure the prediction uncertainty, such as the mutual information between labels and parameters [19], but we found that the prediction variance works better in our experiments.”
- Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network (2020, methodology): “Estimating uncertainty reliably and efficiently has remained an open problem with many important applications such as guiding exploration in Reinforcement Learning (Osband et al., 2016) or as a method for selecting data points for which to acquire labels in Active Learning (Houlsby et al., 2011).”
- Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors (2018, methodology): “Other approaches use classifier confidence (Lewis and Gale, 1994), predicted expected error (Roy and McCallum, 2001), or model disagreement (Houlsby et al., 2011).”
- The Power of Ensembles for Active Learning in Image Classification (2018, background): “Another measure (also known as (BALD, [27]) is the mutual information between data-points and weights.”
- Single-Model Uncertainties for Deep Learning (2018, methodology): “…(entropy), geometrical margin [80, geometrical], ODIN [50], random network distillation [9, distillation], principal component analysis (PCA), Deep Support Vector Data Description [65, SVDD], BALD [40], a random baseline (random), and an oracle trained to separate the in-and out-of-domain examples.”
- Adversarial Phenomenon in the Eyes of Bayesian Deep Learning (2017, methodology): “This has been previously utilised in the context of active learning in the works of [23] and [16].”
- Actively Learning what makes a Discrete Sequence Valid (2017, methodology): “Thus, we design a Bayesian active learning approach for training our model, inspired by classic mutual-information-based approaches (Houlsby et al., 2011; Hern´andez-Lobato et al., 2014).”
- A Survey of Deep Active Learning (2020, methodology): “For its part, BatchBALD [84] opts to expand BALD [67] to the batch query context; this approach no longer calculates the mutual information between a single sample and model parameters, but rather recalculates the mutual information between the batch samples and the model parameters to jointly score”
- Dueling Posterior Sampling for Preference-Based Reinforcement Learning (2019, background): “Other settings include: active learning (Sadigh et al., 2017; Houlsby et al., 2011; Eric et al., 2008), which is focused exclusively on learning an accurate model rather than maximizing utility of decision-making; learning with more structured preference feedback (Radlinski and Joachims, 2005; Shiva”
- Uncertainty Estimation in Bayesian Neural Networks And Links to Interpretability (2018, neutral): “Lastly, if we rearrange the equation and subtract aleatoric uncertainty from predictive uncertainty [33], we have: I [ŷ∗,w|x∗,D] = H [ŷ|x,D]−Ew∼p(w|D) [H [ŷ∗|x∗,w]].”
- Active Learning of Linear Embeddings for Gaussian Processes (2013, methodology): “We employ an active learning strategy due to Houlsby et al. (2011), known as Bayesian active learning by disagreement ( BALD ).”
- Discovering Interpretable Representations for Both Deep Generative and Discriminative Models (2018, methodology): “Our method is similar to Bayesian Active Learning by Disagreement (BALD) (Houlsby et al., 2011).”
- Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2020, methodology): “It is also used to guide methods that require explorations such as active learning [78], continual learning [134], Bayesian optimisation [70], and reinforcement learning [82].”
- Uncertainty in Deep Learning (2016, methodology): “Different acquisition functions exist, and many make use of model uncertainty about the unlabelled data points in order to decide on their potential informativeness [Houlsby et al., 2011].”