Global optimization of a neural network-hidden Markov model hybrid

By Yoshua Bengio, R. Mori, Giovanni Flammia, Ralf Kompe. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Learning: Methods and Applications (2014, background): “Note the use of “hybrid” in 3) above is different from that used sometimes in the literature, which refers to the hybrid systems for speech recognition feeding the output probabilities of a neural network into an HMM (Bengio, 1991; Bengio et al., 1992; Bourlard and Morgan, 1993; Morgan, 2012).”
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results (2014, methodology): “In contrast, some earlier works have proposed to minimize the final decoding error directly by optimizing the sum of costs along the paths in an alignment/decision graph (Bengio et al., 1992; LeCun et al., 1998).”
Jasper: An End-to-End Convolutional Neural Acoustic Model (2019, background): “Hybrid systems combine hidden Markov models to model state dependencies with neural networks to predict states [1, 2, 3, 4].”
End-to-End Text Recognition with Hybrid HMM Maxout Models (2013, methodology): “In other variants of the model, hybrid models are discriminatively trained to optimize segmentation accuracy directly (Bengio et al., 1992; 1995).”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, neutral): “Note the use of “hybrid” in (3) above is different from that used sometimes in the literature, which refers to the hybrid pipeline systems for speech recognition feeding the output probabilities of a neural network into an HMM [31–33].”
Gradient-based learning applied to document recognition (1998, methodology): “This confirms similar results obtained earlier in speech recognition [77].”
Connectionist probability estimators in HMM speech recognition (1994, methodology): “Bridle and Bengio used this approach to optimize the input parameters via some (linear or nonlinear) transform, training the parameters of the HMM by a maximum likelihood process.”
A fusion model of HMM, ANN and GA for stock market forecasting (2007, methodology): “As the optimization criterion 'C' used for the HMM, depends on Y (the output produced by the ANN) it is possible to express 'C' as a function of Y and derive the following equation, using the chain rule (Bengio et al., 1992): oC oY jt ¼ X i oC ob i;t ob i;t oY jt ð5Þ Eq.”
State-of-the-Art Deep Learning: Evolving Machine Intelligence Toward Tomorrow’s Intelligent Network Traffic Control Systems (2017, background): “It is worth noting that the hybrid deep architecture is not the same as feeding the outputs of a traditional neural network to a Hidden Markov Model (HMM) [24]–[26].”
Supervised Sequence Labelling with Recurrent Neural Networks (2012, background): “The two components in a the hybrid can be trained independently, but many authors have proposed methods for combined optimisation (Bengio et al., 1992; Bourlard et al., 1996; Hennebert et al., 1997; Trentin and Gori, 2003) which typically yields improved results.”
Loss Functions for Discriminative Training of Energy-Based Models (2005, background): “Bottou [4] first noted that discriminatively trained HMMs are unduly restricted in their expressive power because of the normalization requirements, a problem recently named “label bias” in [9].”
Deep Neural Networks for Acoustic Modeling in Speech Recognition (2012, background): “However, it has long been known that sequence classification criteria, whic h are more directly correlated with the overall word or phone error rate, can be very helpful in improving recogniti o accuracy [7], [35] and the benefit of using such sequence classification criteria with shallow neural net”