Approximation theory of the MLP model in neural networks

By Allan Pinkus. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “Since the agents’ networks are universal function approximators (Pinkus, 1999), they can represent such a Q a .”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, background): “Since the agents’ networks are universal function approximators (Pinkus, 1999), they can represent such a Qa.”
Breaking the Curse of Dimensionality with Convex Neural Networks (2014, methodology): “We relate these new results to the extensive literature on approximation properties of neural networks (see, e.g., Pinkus, 1999, and references therein) in Section 4.7, and show that these neural networks are indeed adaptive to linear structures, by replacing the exponential dependence in dimension…”
On the Convergence Rate of Training Recurrent Neural Networks (2018, background): “ist certain weights so that the network calculates interesting functions. These results do not usually cover the theoretical issue of how to ﬁnd such weight parameters. We refer interested readers to [15, 45, 51, 53] and the references therein. 2 Notations and Preliminaries Notations. We denote by k”
DGM: A deep learning algorithm for solving partial differential equations (2017, neutral): “Inregards to general results on the approximation power of neural networks we refer the interested reader to classical works [11, 25, 26, 39] and we also mention the recent work by [38], where the authors study the necessary and suﬃcient complexity of ReLU neural networks that is required for…”
The Kolmogorov-Arnold representation theorem revisited (2020, background): “1) can be arbitrarily well approximated by making the network wide, that is, m1,m2 large ([24], Proposition 3.”
Nonparametric regression using deep neural networks with ReLU activation function (2017, background): “An extended summary of the work until the late 90s is given in [39].”
Learning Real and Boolean Functions: When Is Deep Better Than Shallow (2016, methodology): “However, because of the hierarchical structure ofQ, Proposition 2 shows also that a hierarchical network with 9 units can approximate the quadratic expression, and 10 further layers, each with 3 units can approximate the successive powers.”
Are All Layers Created Equal? (2019, background): “More recent research further examines whether deep networks can have superior representation power than shallow ones with the same number of units or edges (Pinkus, 1999; Delalleau and Bengio, 2011; Montufar et al., 2014; Telgarsky, 2016; Shaham et al., 2015; Eldan and Shamir, 2015; Mhaskar and Pogg”
Neural Network with Unbounded Activation Functions is Universal Approximator (2015, unknown): “Refer to Pinkus [12] for more details.”
Shallow Learning For Deep Networks (2018, background): “…is difficult: while generalization, approximation, or optimization results (Barron, 1994; Bach, 2014; Venturi et al., 2018; Neyshabur et al., 2018; Pinkus, 1999) for 1-hidden layer NNs are available, the same studies conclude that multiple-hidden-layer NNs are much more difficult to tackle…”
Relu Deep Neural Networks and Linear Finite Elements (2018, background): “ReLU Deep Neural Networks and Linear Finite Elements 11 and more discussions between Sobolev norms and 9u9 can be found in [8].”
Optimal approximation of piecewise smooth functions using deep ReLU neural networks (2017, background): “In [37], [41] it is shown that assuming a smooth activation function , a shallow network with ε − d/n neurons can uniformly approximate a general C n -function on a d -dimensional set with inﬁnitesimal error ε .”
Approximation rates for neural networks with general activation functions (2019, background): “The first proof of this result can be found in [11] and is summarized in [20].”
Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review (2016, methodology): “In the case of shallow networks, the condition of a smooth activation function can be relaxed to prove density (see [13] , Proposition 3.7): Proposition 2.”
Is Out-of-Distribution Detection Learnable? (2022, background): “[79] If the activation function σ is not a polynomial, then for any continuous function f defined in R, and any compact set C ⊂ R, there exists a fully-connected neural network with architecture q (l1 = d, lg = 1) such that inf fw,b∈F q max x∈C |fw,b(x)− f(x)| < .”