The CAPIO 2017 Conversational Speech Recognition System

By Kyu J. Han, Akshay Chandrashekaran, Jungsuk Kim, Ian Lane. 2017

75 citations recorded by the source corpus.

In this paper we show how we have achieved the state-of-the-art performance on the industry-standard NIST 2000 Hub5 English evaluation set. We explore densely connected LSTMs, inspired by the densely connected convolutional networks recently introduced for image classification tasks. We also propose an acoustic model adaptation scheme that simply averages the parameters of a seed neural network acoustic model and its adapted version. This method was applied with the CallHome training corpus and improved individual system performances by on average 6.1% (relative) against the CallHome portion of the evaluation set with no performance loss on the Switchboard portion. With RNN-LM rescoring and lattice combination on the 5 systems trained across three different phone sets, our 2017 speech recognition system has obtained 5.0% and 9.1% on Switchboard and CallHome, respectively, both of which are the best word error rates reported thus far. According to IBM in their latest work to compare human and machine transcriptions, our reported Switchboard word error rate can be considered to surpass the human parity (5.1%) of transcribing conversational telephone speech.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation (2019, methodology): “Previous results on LibriSpeech using hybrid models are presented in [3, 4].”
Rethinking Evaluation in ASR: Are Our Models Robust Enough? (2020, methodology): “SwitchBoard & Fisher (SB+FSH) consists of conversational telephone speech.”
Transformers with convolutional context for ASR (2019, methodology): “n this paper are orthogonal to further potential improvements to the WER using an LM trained on much larger text corpus. Model LM on extra text dev clean dev other test clean test other CAPIO spk adpt[22] RNNLM 3.12 8.28 3.51 8.58 LSTM[23] 4gramLM 4.79 14.31 4.82 15.30 Gated Cnv[24] 4gramLM 4.6 13.8”
Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions (2019, methodology): “hybrid, speaker adapted CAPIO (single) [33] + RNN 3.”
Fully Convolutional Speech Recognition (2018, methodology): “, pronunciation lexicon) CAPIO (Ensemble) [23] 2.”
Jasper: An End-to-End Convolutional Neural Acoustic Model (2019, methodology): “Traditional N-gram LMs have been augmented with neural LMs in recent work [21, 22, 23].”
Improved Knowledge Distillation via Teacher Assistant (2019, background): “Deep neural networks have achieved state of the art results in a variety of applications such as computer vision (Huang et al. 2017; Hu, Shen, and Sun 2018), speech recognition (Han et al. 2017) and natural language processing (Devlin et al. 2018).”
Model Unit Exploration for Sequence-to-Sequence Speech Recognition (2019, methodology): “Our word-piece model performs better than the previously reported sequenceto-sequence model in [21] while the performance is behind the conventional hybrid system with an n-gram LM [30].”
Improved Knowledge Distillation via Teacher Assistant: Bridging the Gap Between Student and Teacher (2019, background): “Deep neural networks have achieved state of the art results in a variety of applications such as computer vision (Huang et al., 2017; Hu et al., 2018), speech recognition (Han et al., 2017) and natural language processing (Devlin et al., 2018).”
LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring (2019, background): “ 1. Introduction In recent years, language models (LMs) based on long shortterm memory (LSTM) neural networks have become an integral part of many state-of-the-art automatic speech recogition systems [1, 2, 3]. LSTMs thus supersede traditional backoffmodels which are based on word counts. For count ”
An Investigation of Phone-Based Subword Units for End-to-End Speech Recognition (2020, methodology): “9%, outperforming known SWBD results even by large ensemble of systems [41].”
On the limit of English conversational speech recognition (2021, unknown): “We also note that our best 300-hour results could even match system combination results on 2000 hours published only a few years ago [57].”
Advancing Sequence-to-Sequence Based Speech Recognition (2019, background): “We point out that these results are even better than the best hybrid system with lattice rescoring [40].”
The Rwth Asr System for Ted-Lium Release 2: Improving Hybrid Hmm With Specaugment (2020, background): “As shown in the table, the previous best system [24] has a 7.7% WER on the test set.”
Training Neural Speech Recognition Systems with Synthetic Speech Augmentation (2018, unknown): “Recent advances in ASR have further improved upon these models using even more advanced techniques such as replacing n-gram language models with a neural language model, usually in the form of a recurrent neural network (RNN) (Zeyer et al., 2018; Povey et al., 2018; Han et al., 2018).”
Who Needs Words? Lexicon-Free Speech Recognition (2019, background): “Language Model Lexicon dev-clean dev-other test-clean test-other WER CER WER CER WER CER WER CER CAPIO (Ensemble) [38] yes 2.”