Decomposition of Uncertainty for Active Learning and Reliable Reinforcement Learning in Stochastic Systems
By Stefan Depeweg, José Miguel Hernández-Lobato, Finale Doshi-Velez, Steffen Udluft. 2017
27 citations recorded by the source corpus.
Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data. We study in these models a decomposition of predictive uncertainty into its epistemic and aleatoric components. We show how such a decomposition arises naturally in a Bayesian active learning scenario and develop a new objective for reliable reinforcement learning (RL) with an epistemic and aleatoric risk element. Our experiments illustrate the usefulness of the resulting decomposition in active learning and reliable RL.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Andrey Malinin
- Mark John Francis Gales
- Steffen Udluft
- Thomas A. Runkler
- Stefan Depeweg
- Theofanis Karaletsos
- Bruno Mlodozeniec
- José Miguel Hernández-Lobato
Citations in the corpus
- Predictive Uncertainty Estimation via Prior Networks (2018, neutral): “This interpretation was given in [34].”
- Uncertainty Estimation in Autoregressive Structured Prediction (2021, background): “The former is a measure of total uncertainty and the latter is a measure of data uncertainty (Depeweg et al., 2017).”
- Likelihood-free inference with emulator networks (2018, methodology): “We use a rule based on information theory from the active learning literature [25, 26, 27].”
- Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness (2019, background): “This effectively allows a Prior Network to emulate an ensemble and yield the same measures of uncertainty [27, 28], but in closed form and without sampling.”
- Sensitivity analysis for predictive uncertainty (2017, methodology): “To that end we use Bayesian neural networks (BNN) with latent variables [3, 4], a recently introduced probabilistic model that can describe complex stochastic patterns while at the same time account for model uncertainty.”
- Ensemble Distribution Distillation (2019, background): “diversity of the ensemble is lost, which prevents measures of knowledge uncertainty, such as mutual information [22, 23] from being estimated.”