Probabilistic numerics and uncertainty in computations

By Philipp Hennig, Michael A. Osborne, Mark A. Girolami. 2015

331 citations recorded by the source corpus.

We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation with limited time or hardware, are important for much contemporary science and industry. Within applications such as climate science and astrophysics, the need to make decisions on the basis of computations with large and complex data have led to a renewed focus on the management of numerical uncertainty. We describe how several seminal classic numerical methods can be interpreted naturally as probabilistic inference. We then show that the probabilistic view suggests new algorithms that can flexibly be adapted to suit application specifics, while delivering improved empirical performance. We provide concrete illustrations of the benefits of probabilistic numeric algorithms on real scientific problems from astrometry and astronomical imaging, while highlighting open problems with these new algorithms. Finally, we describe how probabilistic numerical methods provide a coherent framework for identifying the uncertainty in calculations performed with a combination of numerical algorithms (e.g. both numerical optimizers and differential equation solvers), potentially allowing the diagnosis (and control) of error sources in computations.

Subfields: Artificial Intelligence, Astronomy and Astrophysics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Astronomy and Astrophysics

Citations in the corpus

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, methodology): “Similar to our approach, Probabilistic Numerical Methods (PNMs) [63, 64, 65] take a statistical point of view of classical numerical methods (e.”
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations (2017, positive): “This approach is aligned in spirit with the emerging field of probabilistic numerics [1], which has roots all the way back to Poincaré’s courses on probability theory [22] and has been recently revived by the pioneering works of [8, 9, 12, 20].”
Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees (2015, methodology): “One advantage of this approach is that we can propagate uncertainty through all subsequent computations to explicitly model the impact of numerical error [15].”
Efficient Acquisition Rules for Model-Based Approximate Bayesian Computation (2017, methodology): “We note that the rationale of our approach is essentially the same as in probabilistic numerics literature (see e.g. Hennig et al. (2015)) or in sequential Bayesian experimental design (see Ryan et al. (2016) for a recent survey).”
Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences (2018, background): “ing deterministic functions is becoming popular in machine learning, in particular in the context of Bayesian optimization (e.g., Bull, 2011) as well as in the emerging eld of probabilistic numerics [Hennig et al., 2015]: For instance, Bayesian quadrature, a probabilistic numerics approach to numeri”
Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems (2016, methodology): “arse discretisation of the PDE and yet produce a meaningful solution to the inverse problem? This paper gives an armative answer by pursuing the development of a probabilistic numerical method (PNM, [37]) for the solution of a generic PDE. The output of a PNM is a probability distribution over the s”
Uniform Coherence (2016, background): “For example, [1] call for the development of probabilistic numerical methods that are uncertain about their calculations; [2] study metareasoning procedures for controlling Monte Carlo simulations by estimating which simulations are likely”
Logical Induction (2016, methodology): “More recently, that thread has been followed by Demski (2012), whose framework we use, and Hutter et al. (2013), who define a probability distribution on logical sentences that is quite desirable, but which admits of no computable approximation (Sawin and Demski 2013). The objective of our algorithm”
Message Passing Neural PDE Solvers (2022, background): “A potential ﬁx would be to fuse this work with that in probabilistic numerics (Hennig et al., 2015), as has been done for RK4 solvers (Schober et al., 2014).”
Bayesian Probabilistic Numerical Methods (2017, methodology): “ovided. The question of whose belief is represented in has been discussed by several authors and a chronology is included in the Electronic Supplement. Of these perspectives we mention in particular Hennig et al. (2015), wherein is the belief of an agent that \we get to design". This oers a con”
Likelihood-free inference with emulator networks (2018, background): “The most principled (but also most costly) rule minimizes the expected integrated variance ( ExpIntVar ) of the approximate posterior after acquiring new data [30, 14].”
Inferring solutions of differential equations using noisy multi-fidelity data (2016, methodology): “This line of work has recently inspired resurgence in probabilistic methods and algorithms [6, 7, 8] that oﬀer a principled and robust handling of uncertainty due to model in-adequacy, parametric uncertainties, and numerical discretization/truncation errors.”