Have Best of Both Worlds: Two-Pass Hybrid and E2E Cascading Framework for Speech Recognition

By Guoli Ye, Vadim Mazalov, Jinyu Li, Yifan Gong. 2021

9 citations recorded by the source corpus.

Hybrid and end-to-end (E2E) systems have their individual advantages, with different error patterns in the speech recognition results. By jointly modeling audio and text, the E2E model performs better in matched scenarios and scales well with a large amount of paired audio-text training data. The modularized hybrid model is easier for customization, and better to make use of a massive amount of unpaired text data. This paper proposes a two-pass hybrid and E2E cascading (HEC) framework to combine the hybrid and E2E model in order to take advantage of both sides, with hybrid in the first pass and E2E in the second pass. We show that the proposed system achieves 8-10% relative word error rate reduction with respect to each individual system. More importantly, compared with the pure E2E system, we show the proposed system has the potential to keep the advantages of hybrid system, e.g., customization and segmentation capabilities. We also show the second pass E2E model in HEC is robust with respect to the change in the first pass hybrid model.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jinyu Li
  2. Dong Yu
  3. Li Deng
  4. Yifan Gong
  5. Zhuo Chen
  6. Chin-Hui Lee
  7. Zhong Meng
  8. Sabato Marco Siniscalchi

In Signal Processing

  1. Jinyu Li
  2. Dong Yu
  3. Yifan Gong
  4. Li Deng
  5. Zhong Meng
  6. Zhuo Chen
  7. Chin-Hui Lee
  8. Sabato Marco Siniscalchi

Citations in the corpus