Exponential ReLU DNN Expression of Holomorphic Maps in High Dimension

By Joost A. A. Opschoor, Christoph Schwab, Jakob Zech. Constructive approximation, 2021

132 citations recorded by the source corpus.

For a parameter dimension $$d\in {\mathbb {N}}$$ d ∈ N , we consider the approximation of many-parametric maps $$u: [-\,1,1]^d\rightarrow {\mathbb R}$$ u : [ - 1 , 1 ] d → R by deep ReLU neural networks. The input dimension d may possibly be large, and we assume quantitative control of the domain of holomorphy of u : i.e., u admits a holomorphic extension to a Bernstein polyellipse $${{\mathcal {E}}}_{\rho _1}\times \cdots \times {{\mathcal {E}}}_{\rho _d} \subset {\mathbb {C}}^d$$ E ρ 1 × ⋯ × E ρ d ⊂ C d of semiaxis sums $$\rho _i>1$$ ρ i > 1 containing $$[-\,1,1]^{d}$$ [ - 1 , 1 ] d . We establish the exponential rate $$O(\exp (-\,bN^{1/(d+1)}))$$ O ( exp ( - b N 1 / ( d + 1 ) ) ) of expressive power in terms of the total NN size N and of the input dimension d of the ReLU NN in $$W^{1,\infty }([-\,1,1]^d)$$ W 1 , ∞ ( [ - 1 , 1 ] d ) . The constant $$b>0$$ b > 0 depends on $$(\rho _j)_{j=1}^d$$ ( ρ j ) j = 1 d which characterizes the coordinate-wise sizes of the Bernstein-ellipses for u . We also prove exponential convergence in stronger norms for the approximation by DNNs with more regular, so-called “rectified power unit” activations. Finally, we extend DNN expression rate bounds also to two classes of non-holomorphic functions, in particular to d -variate, Gevrey-regular functions, and, by composition, to certain multivariate probability distribution functions with Lipschitz marginals.

Subfields: Biomedical Engineering, Computational Mechanics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/344fbf1da8868af6b631155c3e6659370803e1e6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

  1. Xuhui Meng
  2. Zhiping Mao
  3. G. Em Karniadakis
  4. Lu Lu
  5. Zhongqiang Zhang
  6. Shengze Cai

In Computational Mechanics

  1. G. Em Karniadakis
  2. Zhiping Mao
  3. Lu Lu
  4. Zhongqiang Zhang
  5. Shengze Cai
  6. Xuhui Meng
  7. Somdatta Goswami
  8. Maizar Raissi

In Statistical and Nonlinear Physics

  1. G. Em Karniadakis
  2. Lu Lu
  3. Zhiping Mao
  4. Xuhui Meng
  5. Shengze Cai
  6. Zhongqiang Zhang
  7. Somdatta Goswami
  8. Maizar Raissi

Citations in the corpus