The microsoft 2016 conversational speech recognition system

By Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Torsten Bernd Seide, M. Seltzer, Andreas Stolcke, Dong Yu, Geoffry Zweig. 2016

292 citations recorded by the source corpus.

We describe Microsoft's conversational speech recognition system, in which we combine recent developments in neural-network-based acoustic and language modeling to advance the state of the art on the Switchboard recognition task. Inspired by machine learning ensemble techniques, the system uses a range of convolutional and recurrent neural networks. I-vector modeling and lattice-free MMI training provide significant gains for all acoustic model architectures. Language model rescoring with multiple forward and backward running RNNLMs, and word posterior-based system combination provide a 20% boost. The best single system uses a ResNet architecture acoustic model with RNNLM rescoring, and achieves a word error rate of 6.9% on the NIST 2000 Switchboard task. The combined system has an error rate of 6.2%, representing an improvement over previously reported results on this benchmark task.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

Non-local Neural Networks (2017, background): “, non-recurrent) networks for modeling sequences in speech and language [36, 54, 15].”
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour (2017, background): “We are in an unprecedented era in AI research history in which the increasing data and model scale is rapidly improving accuracy in computer vision [22, 40, 33, 34, 35, 16], speech [17, 39], and natural language processing [7, 37].”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “, 2016) and have since been successfully applied to speech recognition (Xiong et al., 2017; Saon et al., 2017; Li et al., 2019b; Wang et al., 2017).”
Aggregated Residual Transformations for Deep Neural Networks (2016, background): “The robustness of VGGnets and ResNets has been proven by various visual recognition tasks [7, 10, 9, 28, 31, 14] and by non-visual tasks involving speech [42, 30] and language [4, 41, 20].”
Gender and Dialect Bias in YouTube’s Automatic Captions (2017, background): “…speech recognition (ASR) has increased substantially over the past decade: a decade ago it was not uncommon to report a ASR error rates of 27% (Sha and Saul, 2007), while a recent Microsoft system achieved a word error rate (WER) of just 6.3% on the Switch-board corpus (Xiong et al., 2016).”
A Wavenet for Speech Denoising (2017, background): “However, until today it has been standard practice not to work directly in the timedomain, but rather to explicitly use time-frequency representations as input [1, 2] – for reducing the high-dimensionality of raw waveforms.”
Achieving Human Parity in Conversational Speech Recognition (2016, background): “This paper is an expanded version of [20], with the following additional material: ar X iv :1 61 0.”
Weighted Transformer Network for Machine Translation (2017, methodology): “, 2017), speech recognition (Xiong et al., 2017; Graves et al., 2013), and machine translation (Wu et al.”
Letter-Based Speech Recognition with Gated ConvNets (2017, background): “On conversational speech (that is not the topic of this paper), the state of the art is still held by complex ConvNets+RNNs acoustic models (which are also trained or reﬁned with a sequence criterion), coupled to domain-adapted language models (Xiong et al., 2017; Saon et al., 2017).”
Training Deep Neural Networks with 8-bit Floating Point Numbers (2018, background): “ning has emerged as the dominant Machine Learning algorithm showing remarkable success in a wide spectrum of applications, including image processing [9], machine translation [21], speech recognition [22] and many others. In each of these domains, Deep Neural Networks (DNNs) achieve superior accurac”
Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling (2017, methodology): “…and vanilla Architecture SWBD CH WER WER Iterated-CTC (Zweig et al., 2016b) 11.3 18.7 BLSTM + LF MMI (Povey et al., 2016) 8.5 15.3 LACE + LF MMI 2 (Xiong et al., 2016b) 8.3 14.8 Dilated convolutions (Sercu and Goel, 2016) 7.7 14.5 Vanilla CTC + Gram-CTC are trained mush faster than vanilla CTC…”
Effects of Talker Dialect, Gender & Race on Accuracy of Bing Speech and YouTube Automatic Captions (2017, background): “This paper expands on that work by reevaluating YouTube’s automatic captions using connected speech and includes an evaluation of a second system, Microsoft’s Bing Speech API.”
Improving Attention-Based End-to-End ASR Systems with Sequence-Based Loss Functions (2018, background): “Hybrid systems improve recognition performance by enhancing each component with deep learning methods[3, 4, 5, 6].”
Attention-Based End-to-End Speech Recognition on Voice Search (2017, background): “n-based speech recognition 1. INTRODUCTION Deep Neural Networks (DNNs) have shown tremendous success and are widely used in speech recognition, usually in combination with Hidden Markov Models (HMMs) [1, 2, 3, 4, 5]. These systems are based on a complicated hybrid of separate components, including a”
CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition (2018, methodology): “to its popularity in research community. Its source code on github obtains 3,748 stars and 1,822 forks [4]. Furthermore, the corpus trained by Kaldi on “Fisher” is also used by IBM [18] and Microsoft [52]. In order to use Kaldi to decode audio, we need a trained model to begin with. There are some m”
The Microsoft 2017 Conversational Speech Recognition System (2017, background): “For details of our implementation and empirical evaluation relative to cross-entropy trained models, see [40].”
Double Forward Propagation for Memorized Batch Normalization (2018, background): “…computer vision (Krizhevsky, Sutskever, and Hinton 2012; He et al. 2016; Szegedy et al. 2017), natural language understanding (Goldberg 2016; Vendrov et al. 2015; Zhou and Yu 2017) and speech recognition (LeCun, Bengio, and Hin-ton 2015; Xiong et al. 2017; Heymann, Drude, and Haeb-Umbach 2016).”
Direct Acoustics-to-Word Models for English Conversational Speech Recognition (2017, background): “Feed-forward, recurrent, and convolutional deep neural networks (DNNs) have signiﬁcantly improved the state-of-the-art in acoustic models (AMs) [1–3].”