Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification

By Rohit Tripathy, Ilias Bilionis. 2018

462 citations recorded by the source corpus.

Abstract State-of-the-art computer codes for simulating real physical systems are often characterized by vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward model evaluations in order to obtain convergent statistics. One, thus, tries to construct a cheap-to-evaluate surrogate model to replace the forward model solver. For systems with large numbers of input parameters, one has to address the curse of dimensionality through suitable dimensionality reduction techniques. A popular class of dimensionality reduction methods are those that attempt to recover a low-dimensional representation of the high-dimensional feature space. However, such methods often tend to overestimate the intrinsic dimensionality of the input feature space. In this work, we demonstrate the use of deep neural networks (DNN) to construct surrogate models for numerical simulators. We parameterize the structure of the DNN in a manner that lends the DNN surrogate the interpretation of recovering a low-dimensional nonlinear manifold. The model response is a parameterized nonlinear function of the low-dimensional projections of the input. We think of this low-dimensional manifold as a nonlinear generalization of the notion of the active subspace. Our approach is demonstrated with a problem on uncertainty propagation in a stochastic elliptic partial differential equation (SPDE) with uncertain diffusion coefficient. We deviate from traditional formulations of the SPDE problem by lifting the assumption of fixed lengthscales of the uncertain diffusion field. Instead we attempt to solve a more challenging problem of learning a map between an arbitrary snapshot of the diffusion field and the response.

Subfields: Computational Theory and Mathematics, Statistical and Nonlinear Physics, Statistics and Probability, Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Statistical and Nonlinear Physics

In Statistics and Probability

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, background): “Applications in flow simulations include pressure projections in solving Navier-Stokes equations [8], fluid flow through random heterogeneous media [9, 10, 11],”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, methodology): “Consequently, there is an effort to fit deep learning models, which have exhibited countless successes across disciplines, as a surrogate for numerical simulations in order to achieve faster model evaluations for UQ that have greater scalability than Gaussian Processes [250].”
Deep Convolutional Encoder‐Decoder Networks for Uncertainty Quantification of Dynamic Multiphase Flow in Heterogeneous Media (2018, methodology): “, 2017, 2018], and surrogate modeling [Tripathy and Bilionis, 2018; Zhu and Zabaras, 2018] due to their robustness and generalization property.”
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data (2019, background): “Deep learning (DL), in particular, the deep neural network (DNN) has become a popular surrogate modeling approach and has shown great potential to deal with high-dimensional nonlinear UQ problems [18–20].”
Recent advances and applications of surrogate models for finite element method computations: a review (2022, unknown): “…reliability Leser et al. (2017) × N/A Structural health monitoring Owen et al. (2017) × × Benchmarking N/A Su et al. (2017) × × Dynamic Gaussian process Structural health monitoring Tripathy and Bilionis (2018) × High dimensional UQ Stochastic PDEs Deng et al. (2020) × Drifted Wiener processes…”
Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks (2019, methodology): “Finally, recent work from [41] overcomes both limitations, by using DNNs to generalize the gradient-free approach to AS by recovering nonlinear manifolds in the input space.”
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, background): “With a similar goal, the ML community has often employed Gaussian Processes as the main technique for quantifying uncertainty in simulating physical processes [34, 229], but neither Gaussian Processes nor reduced models scale well to higher dimensions or larger datasets (Gaussian Processes scale as ”
A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime (2019, methodology): “This is a desirable trait in all surrogates which cannot always be achieved with black-box statistical models that attempt to interpolate the input-output map Tripathy and Bilionis (2018); Zhu and Zabaras (2018).”