Achieving Human Parity in Conversational Speech Recognition

By Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Torsten Bernd Seide, M. Seltzer, Andreas Stolcke, Dong Yu, Geoffry Zweig. 2016

594 citations recorded by the source corpus.

Conversational speech recognition has served as a flagship speech recognition task since the release of the Switchboard corpus in the 1990s. In this paper, we measure the human error rate on the widely used NIST 2000 test set, and find that our latest automated system has reached human parity. The error rate of professional transcribers is 5.9% for the Switchboard portion of the data, in which newly acquainted pairs of people discuss an assigned topic, and 11.3% for the CallHome portion where friends and family members have open-ended conversations. In both cases, our automated system establishes a new state of the art, and edges past the human benchmark, achieving error rates of 5.8% and 11.0%, respectively. The key to our system's performance is the use of various convolutional and LSTM acoustic model architectures, combined with a novel spatial smoothing method and lattice-free MMI acoustic training, multiple recurrent neural network language modeling approaches, and a systematic use of system combination.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Factorization tricks for LSTM networks (2017, methodology): “LSTM networks (Hochreiter & Schmidhuber, 1997) have been successfully used in language modeling (Jozefowicz et al., 2016; Shazeer et al., 2017), speech recognition (Xiong et al., 2016), machine translation (Wu et al., 2016), and many other tasks.”
A Simple Method for Commonsense Reasoning (2018, background): “Although deep neural networks have achieved remarkable successes (e.g., [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]), their dependence on supervised learning has been challenged as a signiﬁcant weakness.”
Unsupervised Pretraining for Sequence to Sequence Learning (2016, background): “More recent acoustic models have found pretraining unnecessary (Xiong et al., 2016; Zhang et al., 2016; Chan et al., 2015), probably because the reconstruction objective of deep belief networks is too easy.”
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine (2017, background): “…of the driving forces behind the recent success of deep learning in challenging tasks, such as object recognition (Krizhevsky et al., 2012), speech recognition (Xiong et al., 2016) and machine translation (Bahdanau et al., 2014), has been the increasing availability of large-scale annotated data.”
Learning Filterbanks from Raw Speech for Phone Recognition (2017, background): “We also focus on MFSC because they are the front-end of state-of-the-art phone [10] and speech [11] recognition systems.”
On Calibration of Modern Neural Networks (2017, methodology): “guage model in speech recognition (Hannun et al., 2014; Xiong et al., 2016), or with camera information for object detection (Kendall & Cipolla, 2016).”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, background): “Deep learning has led to tremendous success in various fields, such as image classification, speech recognition, and game playing [32], [69], [60].”
Certified Defenses against Adversarial Examples (2018, background): “, 2015), speech recognition (Xiong et al., 2016), and playing Go (Silver et al.”
DeepXplore: Automated Whitebox Testing of Deep Learning Systems (2017, background): “Over the past few years, Deep Learning (DL) has made tremendous progress, achieving or surpassing human-level performance for a diverse set of tasks including image classification [31, 66], speech recognition [83], and playing games such as Go [64].”
Opportunities and obstacles for deep learning in biology and medicine (2017, background): “Since 2009 there have been drastic performance improvements with error rates dropping from more than 20% to less than 6% [422] and finally approaching or exceeding human performance in the past year [423,424].”
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, background): “Neural networks have achieved great performance on a wide range of application domains; in particular, they have demonstrated accuracies comparable or better than humans on datasets in the fields of image recognition [10] and speech recognition [29].”
The microsoft 2016 conversational speech recognition system (2016, methodology): “CNN system, RNNLM rescoring yields a relative word error reduction of 20%, and a 16% relative gain for the combined recognition system. (Elsewhere we report further improvements, using LSTM-based LMs [43].) 8. CONCLUSIONS We have described Microsoft’s conversational speech recognition system for 201”
The CAPIO 2017 Conversational Speech Recognition System (2017, background): “With the contributions mainly from convolutional neural networks (CNNs) and recurrent neural networks (RNNs), multiple research groups have continued to improve their system performance on the well-known, industry-standard NIST 2000 Hub5 English evaluation set(1) [1, 2, 3, 4, 5], approaching to the ”
Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize (2018, background): “A complete solution to this problem is likely to have a system on the scale of human parity, which is hard to measure (Hassan et al., 2018; Xiong et al., 2016).”
English Conversational Telephone Speech Recognition by Humans and Machines (2017, methodology): “In [1], the authors claim a human word error rate (WER) of 5.”
Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces (2018, background): “They also became much more reliable, with state-of-the-art speech recognition engines reaching human level in English [56].”
Exploring neural transducers for end-to-end speech recognition (2017, methodology): “Deep neural networks can not only extract acoustic features, which are used as inputs to traditional ASR models like Hidden Markov Models (HMM) [24, 28], but also act as sequence transducers, which results in end-to-end neural ASR systems [2, 6].”
From Eliza to XiaoIce: challenges and opportunities with social chatbots (2018, background): “, 2013, 2015), speech recognition and synthesis (Hinton et al., 2012; Deng et al., 2013; Xiong et al., 2016; Qian et al., 2014; van den Oord et al., 2016), computer vision (Krizhevsky et al.”