Recent Advances in End-to-End Automatic Speech Recognition

By Jinyu Li. 2021

464 citations recorded by the source corpus.

Recently, the speech community is seeing a significant trend of moving from deep neural network based hybrid modeling to end-to-end (E2E) modeling for automatic speech recognition (ASR). While E2E models achieve the state-of-the-art results in most benchmarks in terms of ASR accuracy, hybrid models are still used in a large proportion of commercial ASR systems at the current time. There are lots of practical factors that affect the production model deployment decision. Traditional hybrid models, being optimized for production for decades, are usually good at these factors. Without providing excellent solutions to all these factors, it is hard for E2E models to be widely commercialized. In this paper, we will overview the recent advances in E2E models, focusing on technologies addressing those challenges from the industry's perspective.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Shinji Watanabe
4. Li Deng
5. Tara N. Sainath
6. Takaaki Hori
7. Yifan Gong
8. Rohit Prabhavalkar

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Shinji Watanabe
4. Yifan Gong
5. Tara N. Sainath
6. Li Deng
7. Takaaki Hori
8. Ralf Schlüter

Citations in the corpus

• VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation (2023, neutral): “The primary end-to-end model structures used for ASR (Li, 2022; Prabhavalkar et al., 2023) are connectionist temporal classification (CTC) (Graves et al.”
• Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTS (2023, methodology): “For ASR, we utilize these discrete tokens and their associated texts to train end-to-end (E2E) ASR models [18].”
• A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, unknown): “Pretrained Language Models (PLMs) were primarily employed as a constituent part of Natural language Processing (NLP) systems [1]–[4], such as those used in Speech Recognition [5], [6], Metaphor Processing [7], Sentiment Analysis [8], [9], Information Extraction [10]–[12], and Machine Translation [13”
• End-to-End Speech Recognition: A Survey (2023, background): “Finally, we direct interested readers to Li’s excellent contem-poraneous overview article on end-to-end ASR [344], which offers a complementary perspective to our own.”
• Have Best of Both Worlds: Two-Pass Hybrid and E2E Cascading Framework for Speech Recognition (2021, result): “As a result, compared with hybrid models, E2E models can perform better when there is a large amount of paired audio and text training data, with the condition that the training and test scenarios match well [9].”