Artificial neural networks and their application to sequence recognition

By Yoshua Bengio. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Density estimation using Real NVP (2016, methodology): “This formula has been discussed in several papers including the maximum likelihood formulation of independent components analysis (ICA) [4, 28], gaussianization [14, 11] and deep density models [5, 50, 17, 3].”
NICE: Non-linear Independent Components Estimation (2014, background): “Learning a richer family of transformations was proposed in (Bengio, 1991), but the proposed class of transformations, neural networks, lacks in general the structure to make the inference and optimization practical.”
Deep learning in neural networks: An overview (2014, neutral): “, (Bottou, 1991; Bengio, 1991; Bourlard and Morgan, 1994; Baldi and Chauvin, 1996; Jordan and Sejnowski, 2001; Bishop, 2006; Poon and Domingos, 2011; Dahl et al., 2012; Hinton et al., 2012a).”
Deep Learning: Methods and Applications (2014, methodology): “Note the use of “hybrid” in (3) above is different from that used sometimes in the literature, which refers to the hybrid systems for speech recognition feeding the output probabilities of a neural network into an HMM [17, 25, 42, 261].”
The general inefficiency of batch training for gradient descent learning (2003, background): “Several authors agreed with our findings that on-line training learns more quickly than batch (Bengio, 1991, 1996; Bouttou, Fogelman-Soulie, Blanchet, & Lienard, 1990).”
Learning long-term dependencies with gradient descent is difficult (1994, background): “Recurrent networks are very powerful in their ability to represent context, often outperforming static networks [4].”
3-D Deep Learning Approach for Remote Sensing Image Classification (2018, background): “higher layers in the network as depicted in [43] and [44]”
Convergence Properties of the K-Means Algorithms (1994, background): “Other researchers have compared batch and online algorithms for neural networks, with similar conclusions (Bengio, 1991).”
Acoustic Modeling Using Deep Belief Networks (2012, methodology): “(e-mail: asamir@cs.toronto.edu; gdahl@cs.toronto.edu; hinton@cs.toronto.edu) using a feature vector that describes segments of the temporal evolution of critical-band spectral densities within a single critical band.”
Markovian Models for Sequential Data (2004, background): “The idea of training a set of modules together (rather than separately) with respect to a global criterion with gradient-based algorithms was proposed several years ago [102, 103, 92].”