Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

By Eyke Hüllermeier, Willem Waegeman. Machine-mediated learning, 2019

2,133 citations recorded by the source corpus.

The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this includes the importance of distinguishing between (at least) two different types of uncertainty, often referred to as aleatoric and epistemic. In this paper, we provide an introduction to the topic of uncertainty in machine learning as well as an overview of attempts so far at handling uncertainty in general and formalizing this distinction in particular.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/b631ba962b4403a9c0fd9cce446ef3b1e21ea059

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models (2023, background): “Although differentiating aleatoric vs epistemic uncertainty is interesting, such decomposition is often complex and typically not required for learning algorithms in real-world predictive applications [16].”
Uncertainty Estimation for Language Reward Models (2022, background): “While calibrated total uncertainty is helpful, many applications (such as active learning) depend on separately estimating aleatoric (or data) uncertainty and epistemic (or model) uncertainty [18].”
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “Uncertainty arises from different sources (Hüllermeier and Waegeman, 2021).”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “Since these two types of uncertainties (EPISTEMIC and ALEATORIC) are highly context-dependent, changing the context may blur the difference between them or even cause one to turn into the other.”
A Survey of Machine Unlearning (2022, neutral): “metric that measures how much we know about the optimal hypothesis in the parameter space [72] That is, are we certain that the current model parameters are optimal for a given dataset?).”
A survey of uncertainty in deep neural networks (2021, background): “uncertainty [57]) and model uncertainty (also systemic or epistemic uncertainty [57]).”
A statistical perspective on distillation (2021, background): “Third, the focus on approximating the Bayes probabilities p ⇤ suggests that the teacher’s predictions ought to primarily reﬂect aleatoric, rather than epistemic, uncertainty (Senge et al., 2014; Hüllermeier & Waegeman, 2021).”
Remote patient monitoring using artificial intelligence: Current state, applications, and challenges (2023, background): “The former two uncertainties would lead to uncertainty in the modeling results can be split into data uncertainty (aleatoric uncertainty) and model uncertainty (epistemic uncertainty) (Hüllermeier & Waegeman, 2021).”