Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

By Yinhao Zhu, Nicholas Zabaras. 2018

752 citations recorded by the source corpus.

Abstract We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder–decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data-intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Bayesian inference on millions of uncertain network parameters. This approach achieves state of the art performance in terms of predictive accuracy and uncertainty quantification in comparison to other approaches in Bayesian neural networks as well as techniques that include Gaussian processes and ensemble methods even when the training data size is relatively small. To evaluate the performance of this approach, we consider standard uncertainty quantification tasks for flow in heterogeneous media using limited training data consisting of permeability realizations and the corresponding velocity and pressure fields. The performance of the surrogate model developed is very good even though there is no underlying structure shared between the input (permeability) and output (flow/pressure) fields as is often the case in the image-to-image regression models used in computer vision problems. Studies are performed with an underlying stochastic input dimensionality up to 4225 where most other uncertainty quantification methods fail. Uncertainty propagation tasks are considered and the predictive output Bayesian statistics are compared to those obtained with Monte Carlo estimates.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, methodology): “Taking [9] for an example, the number of training data is often less than 1024.”
Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification (2018, methodology): “[58] use a Bayesian fully convolutional encoder-decoder network to solve an image-to-image regression task mapping the random input coeﬃcient ﬁeld of an elliptic SPDE to the corresponding solution.”
Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks (2019, methodology): “By leveraging recent advances in deep convolutional networks, the authors demonstrate this approach on challenging high-dimensional surrogate modeling tasks in heterogeneous media [42, 43].”
Machine learning in cardiovascular flows modeling: Predicting pulse wave propagation from non-invasive clinical measurements using physics-informed deep learning (2019, methodology): “Leveraging recent advances in physics-informed machine learning [35, 36, 56, 59], we put forth an algorithmic framework for producing physically consistent predictions of ﬂow and pressure wave propagation directly from processing noisy and scattered measurements of blood velocity and wall…”
A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime (2019, methodology): “More recently, deep neural networks (DNNs) LeCun et al. (2015); Goodfellow et al. (2016) have found their way into surrogate modeling of complex computer codes Tripathy and Bilionis (2018); Zhu and Zabaras (2018); Mo et al. (2018); Yang and Perdikaris (2019).”
Integrating Physics-Based Modeling with Machine Learning: A Survey (2020, background): “There are also Bayesian variants of neural networks that consist of distributions of weights and biases [164, 296, 300], but these suffer from high computation times and high dependence on reliable priors.”
Multipole Graph Neural Operator for Parametric Partial Differential Equations (2020, background): “This is in contrast to the state-of-the-art fully convolution network ( FCN ) proposed in [44] which has the property that what it learns is tied to a speciﬁc discretization.”
Choose a Transformer: Fourier or Galerkin (2021, methodology): “Deep convolutional neural networks (DCNN) are widely applied to learn the solution maps with a ﬁxed discretization size [1, 9, 40, 36, 35, 100, 86].”
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (2018, methodology): “To this end, recent advances include using Gaussian process regression [17, 18, 19, 20, 21, 22] and deep neural networks (DNNs) [23, 24, 25, 12, 26] to solve forward problems as well as Bayesian estimation [27] and DNNs [28, 29] for inverse problems.”
Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks (2018, neutral): “Zhu and Zabaras [13] used Stochastic Variational Gradient Descent (SVGD) to perform approximate Bayesian inference on uncertain CNN parameters.”
Data Driven Governing Equations Approximation Using Deep Neural Networks (2018, neutral): “([29, 11, 27, 25]), uncertainty quantification ([49, 54]), etc.”
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data (2019, neutral): “, FC-NN, or CNN [17], as zL(t,x,θ;W,b), i.”
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems (2018, background): “, Bayesian estimation [13] and variational Bayes inference [14], and have been proposed for a wide variety of objectives, from parameter estimation [15] to discovering partial differential equations [16, 17, 18, 19] to learning constitutive relationships [20].”