Links between perceptrons, MLPs and SVMs

By Ronan Collobert, Samy Bengio. International Conference on Machine Learning, 2004

339 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e7e57eacd4b61e684c69703d58255382f78cd7c1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, background): “Between contiguous FCNN layers, the output of a neuron (cell), either the input or hidden cell, is directly passed to and activated by neurons belong to the next layer [80].”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “The Multilayer Perceptrons (MLPs) is the initial Artificial Neural Network (ANN) design, which consists of at least three layers of operations [154].”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “Note that there is a connection between early stopping (see above, choosing the number of training iterations) and L2 regularization (Collobert and Bengio, 2004), with one basically playing the same role as the other (but early stopping allowing a much more efficient selection of the hyper-parameter”
Supervised pattern classification based on optimum‐path forest (2009, methodology): “However, in all SVM approaches, the assumption of separability may also not be valid in any space of finite dimension (Collobert and Bengio, 2004).”
Neural networks designing neural networks: Multi-objective hyper-parameter optimization (2016, neutral): “While there are many different ANN models, the MultiLayer Perceptron (MLP) is a well-known form, which rose in popularity with the advent of the back-propagation training algorithm [5].”
Text summarization using unsupervised deep learning (2017, background): “While a gradient-based algorithm gradually increases weights, corresponding to a regularized learning process (Collobert and Bengio, 2004), early stopping alone is not sufficient enough to prevent over-fitting.”
Deep learning classifiers for hyperspectral imaging: A review (2019, background): “The resulting output can be interpreted as the probability that each input data pattern belongs to a certain class, where the optimization function can be defined as the difference between all the desired outputs yi (for each input data sample xi) and the obtained ones, y i, which can be calculated ”
Learnable Pooling Regions for Image Classification (2013, methodology): “Such joint training of the classiﬁer and the pooling regions can be done by adapting the backpropagation algorithm [Bishop, 1999, LeCun et al., 1998], and so can be interpreted as a densely connected multilayer perceptron [Collobert and Bengio, 2004, Bishop, 1999].”
A Discriminative Kernel-Based Approach to Rank Images from Text Queries (2008, neutral): “This problem (7) can equivalently be written as min w2P T kwk 2 þ C X ðq; p þ ;p À Þ2D train lðw; q; p þ ; p À Þ; where 8ðq; p þ ; p À Þ 2 D train , lðw; q; p þ ; p À Þ ¼ maxð0; 1 À w Á ðq; p þ Þ þ w Á ðq; p À ÞÞ; see [11].”
On the Expressive Power of Deep Neural Networks (2016, background): “inearity. The weights and bias are called W(d) and b(d) respectively. So we have the relations h(d) = W(d)z(d) + b(d); z(d+1) = ˚(h(d)): (1) Our results mostly examine the cases where ˚is a hard-tanh [32] or ReLU nonlinearity. All hard-tanh results carry over to tanh with additional technical steps.”
Learning Smooth Pooling Regions for Visual Recognition (2013, methodology): “Such joint training of the classifier and the pooling regions can be done by adapting the backpropagation algorithm [1] [20], and so can be interpreted as a densely connected multilayer perceptron [1] [4].”
CSI NN: Reverse Engineering of Neural Network Architectures Through Electromagnetic Side Channel (2019, background): “Those layers must consist of nonlinearly activating nodes [7].”
Learning Deep Architectures for AI (2007, methodology): “A simple explanation is based on the observation that stochastic gradient descent with early stopping is similar to aǹ2 regularization of the parameters (Collobert & Bengio, 2004).”
A Survey of Multi-View Representation Learning (2016, neutral): “In contrast to classical SVM, stochastic training is highly scalable and is easy to implement for millions of training examples.”
Systematic Poisoning Attacks on and Defenses for Machine Learning in Healthcare (2015, methodology): “The fifth algorithm is Multilayer Perceptron (MLP), which is based on a feedforward artificial neural network that is trained using backpropagation [33].”