On deep learning as a remedy for the curse of dimensionality in nonparametric regression

By Benedikt Bauer, Michael Kohler. 2019

312 citations recorded by the source corpus.

Assuming that a smoothness condition and a suitable restriction on the structure of the regression function hold, it is shown that least squares estimates based on multilayer feedforward neural networks are able to circumvent the curse of dimensionality in nonparametric regression. The proof is based on new approximation results concerning multilayer feedforward neural networks with bounded weights and a bounded number of hidden neurons. The estimates are compared with various other approaches by using simulated data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

Deep Neural Networks for Estimation and Inference: Application to Causal Effects and Other Semiparametric Estimands (2018, background): “Bach (2017) and Bauer and Kohler (2019) also make important contributions on adaptation properties of deep nets on functions with certain low dimensional structure.”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, background): “Bauer and Kohler (2019) recently demonstrated that least-squares estimates based on FNN can avoid the curse of dimensionality in nonparametric regression.”
A Theoretical Analysis of Deep Q-Learning (2019, methodology): “See Bradtke and Barto (1996); Boyan (2002); Lagoudakis and Parr (2003); Lazaric et al. (2016); Farahmand et al. (2010); Lazaric et al.”
Discussion of: “Nonparametric regression using deep neural networks with ReLU activation function” (2020, background): “Bauer and Kohler (2019) showed that this result even holds for p > 1, provided the sigmoidal function is suitably chosen.”
Posterior Concentration for Sparse Deep Learning (2018, background): “2 n = 1 n Xn i=1 [fb B (xi)− f 0(xi)] 2 is at most a constant multiple away from the minimax rate εn = n−α/(2α+p) (up to a log factor). Unlike related statistical developments (Schmidt-Hieber (2017), Bauer and Kohler (2017)), we approach the reconstruction problem from a purely Bayesian point of vie”
The Kolmogorov-Arnold representation theorem revisited (2020, background): “Approximation properties of deep neural networks for composed functions are studied in [12, 25, 21, 13, 26, 3, 23, 27, 6].”