Approximation capabilities of multilayer feedforward networks

By Kurt Hornik. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Inductive Representation Learning on Large Graphs (2017, neutral): “Moreover, due to our reliance on two universal approximation theorems [15, 29], the required dimensionality is in principle O ( |V| ) .”
Towards Principled Methods for Training Generative Adversarial Networks (2017, background): “…(cid:107) the norm The use of this norm is to make the proofs simpler, but could have been done in another Sobolev norm (cid:107) · (cid:107) 1 ,p for p ∞ covered by the universal approximation theorem in the sense that we can guarantee a neural network approximation in this norm (Hornik, 1991).”
Robust Large Margin Deep Neural Networks (2016, background): “For example, the works in [8], [9] showed that neural networks with a single hidden layer—shallow networks—can approximate any measurable Borel function.”
Deep Clustered Convolutional Kernels (2015, background): “DNNs have reached state of the art performance thanks to their theoretically proven modeling and generalization capabilities (Hornik et al., 1989; Hornik, 1991; Kůrková, 1992), and practically driven by improvements in training algorithms for rapid parameter estimation (Martens, 2010; Sutskever et a”
Conditional Time Series Forecasting with Convolutional Neural Networks (2017, background): “A feedforward neural network requires at least a single hidden layer with a suﬃciently large number of hidden units in order to approximate a non-linear function [13].”
Are Markets Truly Efficient? Experiments Using Deep Learning Algorithms for Market Movement Prediction (2018, neutral): “This goal is supported by the Universal Approximation Theorem, which asserts that a neural net with a single hidden layer, a large number of nodes, with continuous, bounded, and non-constant activation functions can approximate any function arbitrarily closely ([7]), though this has been debated by ”
Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks (2015, background): “Later in [11], it has been established that it was the multilayer feed forward architecture of neural network that gives it the ability of universally approximating arbitrary continuous functions.”
How Powerful are Graph Neural Networks? (2018, methodology): “It can be parameterized by an MLP by the universal approximation theorem (Hornik, 1991).”
GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models (2018, methodology): “We similarly rely on the universal approximation theorem of neural networks (Hornik, 1991).”
Provably Powerful Graph Networks (2019, methodology): “Using MLP universality (Cybenko, 1989; Hornik, 1991) , let m be an MLP so that (cid:107) τ ( x ) − m ( x ) (cid:107) < (cid:15) for all possible colors x ∈ Υ .”
Learning Implicit Fields for Generative Shape Modeling (2018, background): “This is a direct consequence of the universal approximation theorem [20].”
A Papier-Mache Approach to Learning 3D Surface Generation (2018, positive): “sts an integer Ksuch that a multilayer perceptron with ReLU non linearities and Khidden units can approximate Swith a precision . Proof. This is a consequence of the universal representation theorem [16] In the next section, we show how to train such MLPs to align with a desired surface. 4. AtlasNet”
Sorting out Lipschitz function approximation (2018, background): “Classic neural network universality results use constructions which violate the norm-constraints needed for Lipschitz guarantees [14, 24].”
Unsupervised Deep Embedding for Clustering Analysis (2015, background): “To parametrize f θ , deep neural networks (DNNs) are a natu-ral choice due to their theoretical function approximation properties (Hornik, 1991) and their demonstrated feature learning capabilities (Bengio et al., 2013).”
Bitwise Neural Networks (2016, background): “According to the universal approximation theorem, a single hidden layer with a ﬁnite number of units can approximate a continuous function with some mild assumptions (Cybenko, 1989; Hornik, 1991).”