Error estimates for DeepOnets: A deep learning framework in infinite dimensions

By Samuel Lanthaler, Siddhartha Mishra, G. Em Karniadakis. Transactions of Mathematics and Its Applications, 2021

334 citations recorded by the source corpus.

DeepONets have recently been proposed as a framework for learning nonlinear operators mapping between infinite-dimensional Banach spaces. We analyze DeepONets and prove estimates on the resulting approximation and generalization errors. In particular, we extend the universal approximation property of DeepONets to include measurable mappings in non-compact spaces. By a decomposition of the error into encoding, approximation and reconstruction errors, we prove both lower and upper bounds on the total error, relating it to the spectral decay properties of the covariance operators, associated with the underlying measures. We derive almost optimal error bounds with very general affine reconstructors and with random sensor locations as well as bounds on the generalization error, using covering number arguments. We illustrate our general framework with four prototypical examples of nonlinear operators, namely those arising in a nonlinear forced ordinary differential equation, an elliptic partial differential equation (PDE) with variable coefficients and nonlinear parabolic and hyperbolic PDEs. While the approximation of arbitrary Lipschitz operators by DeepONets to accuracy $\epsilon $ is argued to suffer from a ‘curse of dimensionality’ (requiring a neural networks of exponential size in $1/\epsilon $), in contrast, for all the above concrete examples of interest, we rigorously prove that DeepONets can break this curse of dimensionality (achieving accuracy $\epsilon $ with neural networks of size that can grow algebraically in $1/\epsilon $).Thus, we demonstrate the efficient approximation of a potentially large class of operators with this machine learning framework.

Subfields: Environmental Engineering, Mathematical Physics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/0550df65305e0af3b2b20581163a8640b916957a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Environmental Engineering

In Mathematical Physics

In Statistical and Nonlinear Physics

Citations in the corpus

Neural Operator: Learning Maps Between Function Spaces (2021, methodology): “The proof follows the ideas in (Lanthaler et al., 2021) where similar results are obtained for DeepONet(s) on L2(D) by using Lusin’s theorem to restrict the approximation to a large enough compact set and exploit the decay of μ outside it.”
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data (2021, methodology): “In addition to computational tests, we also perform a theoretical comparison of DeepONet versus FNO, following the published work on the theory of DeepONet in [2, 24, 25, 26], and on the more recent theory of FNO in [4, 27].”
Multiwavelet-based Operator Learning for Differential Equations (2021, background): “[46] Samuel Lanthaler, Siddhartha Mishra, and George Em Karniadakis.”
Neural operators for accelerating scientific simulations and design (2023, positive): “The original DeepONet operates on a fixed input grid, but there are later extensions to continuous domains [6, 23].”
On universal approximation and error bounds for Fourier Neural Operators (2021, methodology): “Moreover in this infinite-dimensional setting, universality does not suffice to indicate computational viability or efficiency as the size of the underlying neural networks might grow exponentially with respect to increasing accuracy, see discussion in Lanthaler et al. (2021) on this issue. Hence in”
MIONet: Learning multiple-input operators via tensor product (2022, background): “Recent theoretical works show that DeepONet [6, 19] and FNO [16] may break the curse of dimensionality (CoD) in some problems, and DeepONet can approximate the solution operators of elliptic PDEs with exponential accuracy [32].”