The Microsoft 2017 Conversational Speech Recognition System

By Wayne Xiong, Lingfeng Wu, F. Alleva, Jasha Droppo, Xuedong Huang, Andreas Stolcke. 2017

490 citations recorded by the source corpus.

We describe the latest version of Microsoft's conversational speech recognition system for the Switchboard and CallHome domains. The system adds a CNN-BLSTM acoustic model to the set of model architectures we combined previously, and includes character-based and dialog session aware LSTM language models in rescoring. For system combination we adopt a two-stage approach, whereby acoustic model posteriors are first combined at the senone/frame level, followed by a word-level voting via confusion networks. We also added another language model rescoring step following the confusion network combination. The resulting system yields a 5.1% word error rate on the NIST 2000 Switchboard test set, and 9.8% on the CallHome subset.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dong Yu
2. Jinyu Li
3. Li Deng
4. Tie-Yan Liu
5. Tao Qin
6. Yoshua Bengio
7. Xu Tan
8. Shinji Watanabe

In Speech & Audio

1. Dong Yu
2. Xu Tan
3. Jinyu Li
4. Shinji Watanabe
5. Tao Qin
6. Tie-Yan Liu
7. Yi Ren
8. Zhou Zhao

In Signal Processing

1. Dong Yu
2. Jinyu Li
3. Li Deng
4. Shinji Watanabe
5. Xu Tan
6. M. Seltzer
7. Jasha Droppo
8. Tie-Yan Liu

Citations in the corpus

• An overview of deep learning in medical imaging focusing on MRI (2018, background): “They are, however, not limited to image processing and analysis but are outperforming other approaches in areas like natural language processing [2–4], speech recognition and synthesis [5,6],1 and in the analysis of unstructured, tabular-type data using entity embeddings [7,8].”
• Towards Explainable Deep Neural Networks (xDNN) (2019, background): “Deep learning has demonstrated ability to achieve highly accurate results in different application domains such as speech recognition [2], image recognition [3], and language translation [4] and other complex problems [5].”
• NSML: Meet the MLaaS platform with a real-world case study (2018, background): “(Yamada et al., 2018; Xiong et al., 2018; Liang et al., 2018; Yu et al., 2018; Min et al., 2017) As the applications of the deep learning grow, the complexity of the tasks for designing and tuning the neural network architectures increases as well as the size of models.”
• The CAPIO 2017 Conversational Speech Recognition System (2017, background): “With the contributions mainly from convolutional neural networks (CNNs) and recurrent neural networks (RNNs), multiple research groups have continued to improve their system performance on the well-known, industry-standard NIST 2000 Hub5 English evaluation set(1) [1, 2, 3, 4, 5], approaching to the ”
• Almost Unsupervised Text to Speech and Automatic Speech Recognition (2019, methodology): “TTS and ASR TTS (Arik et al., 2017; Wang et al., 2017; Shen et al., 2018; Ping et al., 2019) and ASR (Chiu et al., 2018; Zhang et al., 2018; Xiong et al., 2018; Yang et al., 2018) (Tjandra et al., 2017; 2018) relied on two well-trained ASR and TTS models to further boost the accuracy with unpaired…”
• Toward Human Parity in Conversational Speech Recognition (2017, methodology): “which significantly boosts our bidirectional long shortterm memory (BLSTM) acoustic model performance 3) The use of long short-term memory (LSTM) rather than recurrent neural nets (RNNs) as language models, and the use of a letter-trigram input representation 4) A two-level system combination, based”
• High-performance brain-to-text communication via handwriting (2021, methodology): “The character error rate decreased to 0.89% and the word error rate decreased to 3.4% averaged across all days, which is comparable to state-of-the-art speech recognition systems with word error rates of 4–5% 14,15 , putting it well within the range of usability.”
• Densely Connected Networks for Conversational Speech Recognition (2018, background): “…networks (CNNs) and recurrent neural networks (RNNs), multiple research groups have continued to improve their system performance on the well-known, industry-standard NIST 2000 Hub5 English evaluation set 1 [1, 2, 3, 4, 5], approaching to the hypothesized human performance of the evaluation set.”
• Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition (2019, background): “, non-recurrent) convolutional neural networks for modeling sequences in speech, language [34, 10, 53, 48], and skeleton [18, 58, 30, 51] due to their superior performance.”
• Acoustic Modeling for Overlapping Speech Recognition: Jhu Chime-5 Challenge System (2019, background): “Recently, super-human performance has been achieved on conversational telephone speech using publicly available datasets [3, 4, 5].”
• CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition (2018, methodology): “Besides, the corpus trained by Kaldi on “Fisher” is also used by IBM [34] and Microsoft [50].”
• Long short-term memory networks in memristor crossbar arrays (2018, methodology): “L σ δ ̂ = ∂ ∂ ̂ = ′ − y y y y ( ) (9) t t t t target Table 2 | Summary of the uSF-NiSt gait database subset”
• LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring (2019, background): “In recent years, language models (LMs) based on long short-term memory (LSTM) neural networks have become an integral part of many state-of-the-art automatic speech recogition systems [1, 2, 3].”
• Adversarial Robustness vs. Model Compression, or Both? (2019, background): “ or deep neural networks (DNNs) have achieved extraordinary performance in many application domains such as image classification [22, 46], object detection and recognition [31, 42], speech recognition [24, 53], natural language processing [10, 38], medical image analysis [32, 44], and disease diagnos”
• On the limit of English conversational speech recognition (2021, background): “As shown by [34, 35], significant gain has been observed on the Switchboard test sets when the language model (LM) is trained at the conversation level.”
• Hello Edge: Keyword Spotting on Microcontrollers (2017, background): “Deep learning algorithms have evolved to a stage where they have surpassed human accuracies in a variety of cognitive tasks including image classification [1] and conversational speech recognition [2].”
• Deep convolutional neural networks for eigenvalue problems in mechanics (2018, unknown): “recently reached human-level accuracy [8, 9].”