Approximation and estimation bounds for artificial neural networks

By Andrew R. Barron. Machine-mediated learning, 2004

454 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/722c52711a8014694d68839f0ffc52ba8f7fc621

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks (2017, background): “Although the expressive ability of neural networks is relatively well-understood [5], computational tractability of training such”
Long short-term memory - Fully connected (LSTM-FC) neural network for PM2.5 concentration prediction. (2019, background): “Neural networks have a strong ability of non-linear expression (Barron (1994)) that is why they are suitable to act as spatial combinators.”
Risk bounds for mixture density estimation (2005, background): “For neural networks and projection pursuit, approximation and estimation bounds can be found in [1, 2, 5, 11].”
Aggregating regression procedures to improve performance (2004, background): “This phenomenon is closely related to the advantage of sparse approximations as observed in wavelet estimation (see Donoho and Johnstone 1998), neural networks and subset selection (Barron 1994; Yang and Barron 1998; Barron et al., 1999).”
The Expressive Power of Neural Networks: A View from the Width (2017, background): “The celebrated universal approximation theorem states that depth-2 networks with suitable activation function can approximate any continuous function on a compact domain to any desired accuracy [3] [1] [9] [6].”
Safe Control With Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction Methods for Robotics and Control (2022, background): “By increasing the size of these networks, this representation can approximate any continuous function [45], [46], alleviating limitations due to the choice of function space.”
On the Power of Over-parametrization in Neural Networks with Quadratic Activation (2018, background): “The expressive ability of shallow neural network dates back to 90s (Barron, 1994).”
Parametric or nonparametric? A parametricness index for model selection (2011, background): “Publications of work in this direction include [3, 4, 5, 6, 10, 14, 15, 22, 23, 24, 43, 71], to name a few.”
Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality (2018, methodology): “Function class Hölder Barron class m-Sobolev (0 < β ≤ 2) m-Besov (0 < β) Approximation Author Yarotsky (2017), Liang & Srikant (2016) Barron (1993) Montanelli & Du (2017) This work”
Spectral-Pruning: Compressing deep neural network via spectral analysis (2018, background): “, [28, 5, 36, 6, 26, 27, 42, 25, 46], but they are not about model compression.”
Depth Separation for Neural Networks (2017, background): “In particular, separation for deep networks [11, 10], expressive power of depth two networks [4, 8, 7, 2], and more [5, 3].”
Capacity of reproducing kernel spaces in learning theory (2003, background): “Therefore, the setting we consider here is different from those in [ 4 ], [15] and the upper bounds for the covering number are needed for our purposes.”
On deep learning as a remedy for the curse of dimensionality in nonparametric regression (2019, background): “The rate of convergence has been analyzed by Barron (1991, 1993, 1994), McCa(cid:27)rey and Gallant (1994) and Kohler and Krzy»ak (2005, 2017).”
Optimal approximation of piecewise smooth functions using deep ReLU neural networks (2017, background): “Other works analyze the necessary and sufficient size of networks to approximate certain classes of functions, whose Fourier transform has a bounded first moment [4, 3].”
Greedy Layerwise Learning Can Scale to ImageNet (2018, methodology): “Secondly, understanding the link between shallow Neural Networks (NNs) and deep NNs is difficult: while generalization, approximation, or optimization results (Barron, 1994; Bach, 2014; Venturi et al., 2018; Neyshabur et al., 2018; Pinkus, 1999) for 1-hidden layer NNs are available, the same studies”
Neural network approximation (2020, methodology): “One celebrated model class of this type was introduced by Andrew Barron in (Barron 1994).”
Learning Polynomials with Neural Networks (2014, methodology): “We note here we require the target function to be a k-sparse degree-d polynomial, more restrictive than requiring the target to be representable by a small network as in Barron (1994).”
Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods (2017, methodology): “The polynomial factors for sample complexity in our estimation error are slightly worse than the bound provided in Barron (1994), but note that we provide an estimation method which is both computationally and statistically efficient, while the method in Barron (1994) is not computationally…”