Universal approximation bounds for superpositions of a sigmoidal function

By Andrew R. Barron. 1993

Subfields: Statistical Learning Theory, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

In Information Theory

Citations in the corpus

Theoretical Guarantees for Model Auditing with Finite Adversaries (2019, background): “In particular, our main results heavily rely on the seminal work of [16] regarding the rate (with respect to the number of neurons) at which 2-layer neural networks approximate certain functions.”
Deep Neural Networks for Estimation and Inference: Application to Causal Effects and Other Semiparametric Estimands (2018, methodology): “, White, 1989, 1992; Schmidt-Hieber, 2017; Liang, 2018; Bauer and Kohler, 2017, in specific scenarios). Chen and Shen (1998) and Chen and White (1999) share our goal, fast convergence rates for use in semiparametric inference, but focus on shallow, sigmoid-based networks compared to our deep, ReLU-b”
Convergence Analysis of Two-layer Neural Networks with ReLU Activation (2017, background): “It is well known that neural networks have great expressive power [22, 7, 3, 8, 31].”
Generalization in Deep Learning (2017, background): “, 1993) and the exponential advantage against hand-crafted features (Barron, 1993).”
Diverse Neural Network Learns True Target Functions (2016, methodology): “Using learned weights leads to considerable smaller models as shown in [Barron, 1993].”
ℓ1-regularized Neural Networks are Improperly Learnable in Polynomial Time (2015, background): “It is known that any smooth function can be approximated by a network with just one hidden layer [4], but training such a network is NP-hard [6].”
FNet: Mixing Tokens with Fourier Transforms (2021, background): “For example, universal approximation properties of neutral networks are typically investigated by first showing that a particular class of neutral networks is dense in the space of Fourier series, which in turn are dense in the relevant Hilbert space; see, for example, (Cybenko, 1989; Barron, 1993).”
On the Spectral Bias of Neural Networks (2018, methodology): “Barron (1993) makes use of Fourier space properties of the target function to derive an architecture-dependent approximation bound.”
Risk bounds for mixture density estimation (2005, background): “For neural networks and projection pursuit, approximation and estimation bounds can be found in [1, 2, 5, 11].”
On the Complexity of Learning Neural Networks (2017, background): “It is well-known that Neural Networks (NN’s) provide universal approximate representations [11, 6, 2] and under mild assumptions, i.”
Orthogonal Deep Neural Networks (2019, background): “Classical theories of DNNs show that they are universal approximators [6], [25].”