Recent advances in deep learning for speech research at Microsoft

By Li Deng, Jinyu Li, Jui-Ting Huang, Kaisheng Yao, Dong Yu, Frank Torsten Bernd Seide, M. Seltzer, Geoffry Zweig, Xiaodong He, Jason Williams and 2 more. 2013

820 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dong Yu
2. Li Deng
3. Jianfeng Gao
4. Jinyu Li
5. Yoshua Bengio
6. Mohit Bansal
7. Kyunghyun Cho
8. Quoc V. Le

In Natural Language Processing

1. Jianfeng Gao
2. Li Deng
3. Mohit Bansal
4. Yoshua Bengio
5. Dong Yu
6. Kyunghyun Cho
7. Xiaodong He
8. Richard Socher

In Signal Processing

1. Dong Yu
2. Jinyu Li
3. Li Deng
4. Yifan Gong
5. Shinji Watanabe
6. Yoshua Bengio
7. S. Renals
8. Tara N. Sainath

Citations in the corpus

• Adam: A Method for Stochastic Optimization (2014, methodology): “SGD proved itself as an efficient and effective optimization method that was central in many machine learning success stories, such as recent advances in deep learning (Deng et al., 2013; Krizhevsky et al., 2012; Hinton & Salakhutdinov, 2006; Hinton et al., 2012a; Graves et al., 2013).”
• Deep Learning: Methods and Applications (2014, methodology): “The success was quickly extended to large vocabulary speech recognition tasks with hundreds and even thousands of hours of training set and with thousands of tri-phone states, including the Switchboard and Broadcast News databases, and Google’s voice search and YouTube tasks [94, 161, 184, 309, 311,”
• Recurrent Reinforcement Learning: A Hybrid Approach (2015, background): “For example, Deng et al. (2013) apply a deep network to track belief states in a spoken dialogue system.”
• From Eliza to XiaoIce: challenges and opportunities with social chatbots (2018, background): “, 2013, 2015), speech recognition and synthesis (Hinton et al., 2012; Deng et al., 2013; Xiong et al., 2016; Qian et al., 2014; van den Oord et al., 2016), computer vision (Krizhevsky et al.”
• A tutorial survey of architectures, algorithms, and applications for deep learning (2014, background): “The most notable groups include all major industrial speech labs worldwide: Microsoft [155, 156, 168, 169], IBM [125, 141, 175], Google [120, 176], and Baidu.”
• ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition (2018, methodology): “…manual step of feature engineering and representation required in conventional ML. Due to this powerful advantage, deep learning has gained significant attention in the field of computer science with breakthrough results in computer vision 48,49 , speech recognition 50,51 and text processing 52 .”
• Bidirectional Helmholtz Machines (2015, background): “It is widely used for applications including visual object recognition, speech perception, language comprehension, and more [23, 24, 35, 93].”
• Speech emotion recognition using deep neural network and extreme learning machine (2014, background): “In addition, motivated by the recent progress in speech recognition [16, 17], we have attempted to train the DNN directly using the filterbank or spectral features, but the performance is not satisfactory.”
• Foundations and Advances in Deep Learning (2014, methodology): “3 Microsoft has replaced the existing speech recognition algorithm based on Gaussian mixtures with one based on deep neural networks (Deng et al., 2013).”
• Investigation of unsupervised adaptation of DNN acoustic models with filter bank input (2014, background): “In particular, log mel-filter bank channel outputs, or FBANK features, have been shown to provide improved recognition accuracy [2, 4].”
• Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition (2015, methodology): “The primary use of the CNN mainly lies in 2D image [Krizhevsky et al., 2012; Zeiler and Fergus, 2014], 3D videos [Ji et al., 2010] and speech recognition [Deng et al., 2013].”
• Brain-Inspired Classroom Occupancy Monitoring on a Low-Power Mobile Platform (2014, methodology): “Convolutional networks are also being used by Google [1] and Microsoft [8] to power various tasks such as visual search algorithms and speech recognition.”