Multilayer feedforward networks are universal approximators

By Kurt Hornik, Maxwell B. Stinchcombe, Halbert White. 1989

Subfields: Model Architectures, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Scaling & Learning Theory

Citations in the corpus

On the Number of Linear Regions of Deep Neural Networks (2014, background): “A well-known result states that a feedforward neural network with a single, huge, hidden layer is a universal approximator of Borel measurable functions (see Hornik et al. 1989, Cybenko 1989).”
Neural Tangent Kernel: Convergence and Generalization in Neural Networks (2018, background): “[7, 10] for classical theorems about ANNs and approximation).”
Explaining and Harnessing Adversarial Examples (2014, background): “The universal approximator theorem (Hornik et al., 1989) guarantees that a neural network with at least one hidden layer can represent any function to an arbitary degree of accuracy so long as its hidden layer is permitted to have enough units.”
A Closer Look at Memorization in Deep Networks (2017, background): “On the contrary, deep nets are known to be universal approximators, capable of representing arbitrarily complex functions given sufficient capacity (Cybenko, 1989; Hornik et al., 1989).”
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (2017, background): “This assumption is often referred to as the nonparametric limit (Goodfellow et al., 2014) and is justified by the fact that deep neural networks are universal function approximators (Hornik et al., 1989).”
Least Squares Generative Adversarial Networks (2016, background): “As stated in [9], there should be a deterministic relationship between input and output.”
How to Construct Deep Recurrent Neural Networks (2013, background): “However, this highly non-linear transition can be modeled by an MLP with one or more hidden layers which has an universal approximator property (see, e.g., Hornik et al. , 1989).”
Neural Collaborative Filtering (2017, background): “The neural network has been proven to be capable of approximating any continuous function [17], and more recently deep neural networks (DNNs) have been found to be eﬀective in several domains, ranging from computer vision, speech recognition, to text processing [5, 10, 15, 47].”
Theoretical Guarantees for Model Auditing with Finite Adversaries (2019, background): “The latter results characterize how well certain functions can be approximated using neural networks, see [16, 17, 20–25] and references therein.”
Wireless Network Intelligence at the Edge (2018, background): “Here, n is the sample complexity to satisfy a target generalization error with a target reliability 1 − ε. PAC Framework Applications: (a) PAC-VC Bound: If the hypothesis space H is infinite, e.g., in deep NNs or non-parametric models such as GP, one can derive the GE using the VapnikChervonenkis (V”
The Limitations of Deep Learning in Adversarial Settings (2015, background): “states one hidden layer is sufficient to represent arbitrarily accurately a function [20].”
Bayesian Nonparametric Federated Learning of Neural Networks (2019, methodology): “The model we have presented thus far can handle any arbitrary width single layer neural network, which is known to be theoretically sufficient for approximating any function of interest (Hornik et al., 1989).”
Deep Learning for Distributed Optimization: Applications to Wireless Resource Management (2019, background): “Proposition 1, which is based on the universal approximation theorem [29], implies that for a given set A , there exist a set parameter θ C such that the associated DNN of the structure in (8) can approximate any continuous function z C ( a ) with arbitrary small positive error ε .”
Deep learning in neural networks: An overview (2014, background): “Many practitioners found solace in a theorem (Kolmogorov, 1965a; Hecht-Nielsen, 1989; Hornik et al., 1989) stating that an NN with a single layer of enough hidden units can approximate any multivariate continous function with arbitrary accuracy.”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, background): “As was mathematically demonstrated [5] in 1989, a multilayer feed-forward ANN (MLP) is an universal approximator, hence their usefulness in classiﬁcation and regression problems.”