Regularization Theory and Neural Networks Architectures

By Federico Girosi, Michael J. Jones, Tomaso A. Poggio. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Fastfood: Approximate Kernel Expansions in Loglinear Time (2014, background): “This allows us to identify P † P from k and vice versa (Smola et al., 1998a; Girosi, 1998; Girosi et al., 1995; Girosi and Anzellotti, 1993; Wahba, 1990).”
Recurrent Neural Networks Approach to the Financial Forecast of Google Assets (2017, positive): “It follows, that it is better to use early stopping technique, see [13], to know when optimally stop the training scheme.”
Do Deep Generative Models Know What They Don't Know? (2018, background): “Dating back at least to (Girosi et al., 1995), penalizing the Frobenius norm of a neural network’s Jacobian—which upper bounds the volume term 3 —has been shown to improve generalization.”
Knowledge Transfer in Deep Block-Modular Neural Networks (2015, background): “These successes can mainly be attributed to the use of pre-training [7,21], sharing [16], and various forms of regularization [8,10], as well as to increases in computational capacity and data availability [22].”
Structural Modelling with Sparse Kernels (2002, background): “Girosi et al. (1995) shows that additive models can be formulated as regularisation networks, thereby allowing additive regularisers to be constructed.”
An Evolutionary Algorithm for Large-Scale Sparse Multiobjective Optimization Problems (2020, background): “First, many sparse SOPs embed the sparsity of solutions in the objective function [15], [39], which can actually be regarded as the second objective to be optimized; that is, sparse SOPs can be converted into sparse MOPs.”
Grafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space (2003, methodology): “Ω 2 This regularizer is seen in ridge regression (Hoerl and Kennard, 1970), the support vector machine (Boser et al., 1992, Schölkopf and Smola, 2002) and regularization networks (Girosi et al., 1995).”
Advances in Large Margin Classifiers (2000, methodology): “, 1995] were typically used with shift invariant stabilizers (tensor product and additive stabilizers are exceptions, see [Girosi et al., 1995]).”
Adversarial Examples on Object Recognition (2020, background): “Including adversarial examples in the training data set is a form of regularization [60] i.e. a strategy designed to reduce the test error, possibly aﬀecting the training error.”
Gene networks inference using dynamic Bayesian networks (2003, background): “…parametrized by θ (k) , is defined as the expectation (operator E[·]) of the penalized log-likelihood (Ormoneit et al., 1998 Machine learning theory ensures that choosing a prior that controls the complexity of the learned model produces a more efficient model in prediction (Girosi et al., 1995).”
Hybrid Models with Deep and Invertible Features (2019, unknown): “This optimization objective stands in direct contrast to a long history of gradient-based regularization penalties (Girosi et al., 1995; Bishop, 1995; Rifai et al., 2011).”
Kernel independent component analysis (2003, methodology): “For concreteness we restrict ourselves to translation-invariant kernels in this paper; that is, to kernel functions of the form K(x; y )= k(x y). In this case the RKHS can be described succinctly using Fourier theory ( Girosi et al., 1995, Smola et al., 1998).”