Binary coding of speech spectrograms using a deep auto-encoder

By Li Deng, M. Seltzer, Dong Yu, A. Acero, Abdel-rahman Mohamed, Geoffrey E. Hinton. 2010

385 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Li Deng
2. Dong Yu
3. Yoshua Bengio
4. Jinyu Li
5. Jianfeng Gao
6. Mohit Bansal
7. Kyunghyun Cho
8. Quoc V. Le

In Natural Language Processing

1. Li Deng
2. Dong Yu
3. Yoshua Bengio
4. Jianfeng Gao
5. Kyunghyun Cho
6. Xiaodong He
7. Mohit Bansal
8. Richard Socher

In Signal Processing

1. Li Deng
2. Dong Yu
3. Jinyu Li
4. Yifan Gong
5. Yoshua Bengio
6. Junichi Yamagishi
7. Shinji Watanabe
8. Tara N. Sainath

Citations in the corpus

• Deep Generative Stochastic Networks Trainable by Backprop (2013, background): “However, the most impressive recent results have been obtained with purely supervised learning techniques for deep networks, in particular for speech recognition (Dahl et al., 2010; Deng et al., 2010; Seide et al., 2011) and object recognition (Krizhevsky et al., 2012).”
• GSNs : Generative Stochastic Networks (2015, background): “recognition (Dahl et al., 2010; Deng et al., 2010; Seide et al., 2011) and object recognition (Krizhevsky et al.”
• Representation Learning: A Review and New Perspectives (2012, background): “…deep learning, and representation learning has had a strong impact in the area of speech recognition, with breakthrough results (Dahl et al., 2010; Deng et al., 2010; Seide et al., 2011a; Mohamed et al., 2012; Dahl et al., 2012; Hinton et al., 2012) obtained by several academics as well as…”
• A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, background): “2) [44, 45] are mostly adopted for educational uses, as McCulloch and Pitts [1] cells are still used in this context.”
• A tutorial survey of architectures, algorithms, and applications for deep learning (2014, methodology): “A similar deep auto-encoder architecture described in Section IV and in (Deng et al., 2010) is used but it can be considered as a generalization from a single modality to two modalities.”
• Comparing multilayer perceptron to Deep Belief Network Tandem features for robust ASR (2011, methodology): “this problem and have been applied in several fields such as computer vision (see [6]), phone classification (see [7, 3]), and speech coding (see [8]).”
• Modeling spectral envelopes using restricted Boltzmann machines for statistical parametric speech synthesis (2013, methodology): “Finally, the context-dependent RBM-HMMs can be constructed for modeling the spectral envelopes.”
• General Stochastic Networks for Classification (2014, background): “Deep models in representation learning, also obtain impressive results in supervised learning problems, such as speech recognition, e.g. [7, 8, 9] and computer vision tasks [10].”
• Person Re-identification: Past, Present and Future (2016, methodology): “As another promising idea, it is worth trying to pre-train CNN models using the detection/tracking data using the auto-encoder [143] or the generative adversarial nets (GAN) [144].”
• Learning with Recursive Perceptual Representations (2012, background): “This gives rise to multiple layers in unsupervised feature learning, as exemplified in Deep Belief Networks [14, 13, 9], layered Convolutional Neural Networks [15], Deep Auto-encoder [14, 9], etc. Applications of such stacking methods includes object recognition [15, 26, 4], speech recognition [20],”
• Reverberant speech recognition combining deep neural networks and deep autoencoders augmented with a phone-class feature (2015, methodology): “Recognition of reverberant speech is performed combining “standard” DNN-HMM [14] decoding and a feature enhancement through deep autoencoder (DAE) [9, 10, 15].”
• On Nonparametric Guidance for Learning Autoencoder Representations (2011, methodology): “Neural networks, for example, provide such feed forward feature extractors, and autoencoders, specifically, have found use in domains such as image classification (Vincent et al., 2008), speech recognition (Deng et al., 2010) and Bayesian nonparametric models (Adams et al., 2010).”
• A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends (2019, methodology): “The approach has been successfully applied to encode documents for faster subsequent retrieval [81], image retrieval, efficient speech features [82] etc.”