Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems

By Youssef M. Marzouk, Habib N. Najm. Journal of Computational Physics, 2008

444 citations recorded by the source corpus.

We consider a Bayesian approach to nonlinear inverse problems in which the unknown quantity is a spatial or temporal field, endowed with a hierarchical Gaussian process prior. Computational challenges in this construction arise from the need for repeated evaluations of the forward model (e.g., in the context of Markov chain Monte Carlo) and are compounded by high dimensionality of the posterior. We address these challenges by introducing truncated Karhunen-Loeve expansions, based on the prior distribution, to efficiently parameterize the unknown field and to specify a stochastic forward problem whose solution captures that of the deterministic forward model over the support of the prior. We seek a solution of this problem using Galerkin projection on a polynomial chaos basis, and use the solution to construct a reduced-dimensionality surrogate posterior density that is inexpensive to evaluate. We demonstrate the formulation on a transient diffusion equation with prescribed source terms, inferring the spatially-varying diffusivity of the medium from limited and noisy data.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e3bccdc42be75d9a6478bf685e56cf5a49cf0185

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

Citations in the corpus

A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems (2015, unknown): “Stochastic spectral approximations have been used both in forward propagation of uncertainty [107, 226] and for statistical inverse problems [168, 169].”
Surrogate accelerated sampling of reservoir models with complex structures using sparse polynomial chaos expansion (2015, methodology): “[56] applied the polynomial chaos expansion along with the MCMC algorithm in the Bayesian framework for a low-dimensional problem and proved the efficiency of the algorithm.”
Stochastic Collocation Algorithms Using l1-Minimization for Bayesian Solution of Inverse Problems (2015, methodology): “2) is called the stochastic surrogate model [25, 26, 27, 28].”
A stochastic collocation approach to Bayesian inference in inverse problems (2009, methodology): “The gPC stochastic Galerkin approach has also been extended to Bayesian inference of spatially-distributed quantities, such as inhomogeneous material properties appearing as coefﬁcients in a PDE [16].”
Survey of multifidelity methods in uncertainty propagation, inference, and optimization (2018, methodology): “MCMC methods are a popular way to sample from the posterior distribution and have been studied extensively [178, 90, 139, 80, 50, 137, 53, 52]; see also the books [77, 130].”
Confronting the Challenge of Modeling Cloud and Precipitation Microphysics (2020, background): “Other emulation approaches include polynomial chaos expansion (Iskandarani et al., 2016; Marzouk & Najm, 2009; Sraj et al., 2016) and multivariate adaptive regression splines (MARS; Friedman, 1991; Friedman & Roosen, 1995).”
Structured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion (2018, methodology): “Surrogate models such as polynomial chaos [8, 9], proper orthogonal decomposition [10], or Gaussian processes [11, 12] have been studied.”
Efficient posterior exploration of a high‐dimensional groundwater model from two‐stage Markov chain Monte Carlo simulation and polynomial chaos expansion (2013, methodology): “Note that this dimensionality reduction technique is commonly employed to represent Gaussian spatial fields such as observed for the log conductivity, and several studies have used this approach in conjunction with MCMC simulation [e.g., Dostert et al., 2006, 2009; Marzouk and Najm, 2009].”
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion (2012, methodology): “Because the total number of terms in the PC expansion is combinatorial in the parameter dimension, a truncated Karhunen–Loève (KL) expansion (based on the prior distribution) may be employed to prevent the cost of the stochastic forward problem from becoming prohibitive [31].”
Adaptive Gaussian Process Approximation for Bayesian Inference with Expensive Likelihood Functions (2017, methodology): “xamples illustrate that the proposed method can substantially improve the performance of the GP approximation. We note that, other surrogate models, notably the generalized polynomial chaos expansion [14, 16, 17, 18, 20], have also been used to accelerate the Bayesian computation. The comparison of ”
The statistical finite element method (statFEM) for coherent synthesis of observation data and model predictions (2019, methodology): “Usually, the ψ(x) are chosen as either the Lagrange, B-spline, Karhunen-Loeve or Gaussian process basis functions, see [13, 43, 45].”
Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [Chapter 7] (2010, methodology): “In particular, one can introduce a Karhunen-Loève (K-L) expansion based on the prior covariance, transforming the inverse problem to inference on a truncated sequence of weights of the K-L modes (Marzouk and Najm 2009).”
Bayesian Solution Uncertainty Quantification for Differential Equations (2013, background): “…and in particular those based on mathematical descriptions of complex phenomena such as the weather, ocean currents, ice sheet ﬂow, and cellular protein transport (Ghanem and Spanos, 2003; Kaipio et al., 2004; Huttunen and Kaipio, 2007; Marzouk et al., 2007; Marzouk and Najm, 2009; Stuart, 2010).”
Simulation-based optimal Bayesian experimental design for nonlinear systems (2011, methodology): “, [56, 57, 58]), and this is the approach we shall use.”
Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems (2018, unknown): “Many surrogate modeling methods have been successfully employed in this context, for example, projection-type reduced order models [2, 11, 12, 16, 21], polynomial chaos (PC) [24, 26, 27, 28, 41] and Gaussian process regression [18, 36, 38], to name a few.”