Original Contribution: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function

By M. Leshno, V. Lin, Allan Pinkus, S. Schocken. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

Citations in the corpus

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (2018, background): “While it has long been known that ANNs can approximate any function with sufficiently many hidden neurons [10, 12], it is not known what the optimization of ANNs converges to.”
End-to-end Optimized Image Compression (2016, background): “As such, the observed efﬁciency advantage relative to pointwise nonlinearities is expected, and a variant of a universal function approximation theorem (e.g., Leshno et al., 1993) should hold.”
Breaking the Curse of Dimensionality with Convex Neural Networks (2014, background): “The result below notably shows the density of G1 in uniform norm in the space of Lipschitz-continuous functions of the given parity, which is already known since our activation functions are not polynomials (Leshno et al., 1993).”
Generalization in Deep Learning (2017, background): “For expressivity of neural networks, there are classical results regarding their universality (Leshno et al., 1993) and exponential advantages over hand-crafted features (Barron, 1993).”
On the Spectral Bias of Neural Networks (2018, background): “Hornik et al. (1989); Cybenko (1989); Leshno et al. (1993) have shown that neural networks can be universal approximators when given sufficient width; more recently, Lu et al. (2017) proved that this property holds also for width-bounded networks.”
TI-POOLING: Transformation-Invariant Pooling for Feature Learning in Convolutional Neural Networks (2016, background): “These advances are largely attributed to expressiveness of deep neural networks with many parameters, that are effectively able to approximate any decision function in the data space [18].”
On the Computational Power of Winner-Take-All (2000, background): “According to ( Leshno et al., 1993 ) there exist ~ n 2 N; ~ j 2 R, and -gates G1j for j = 1; : : : ; ~ n so that”