Acoustic Modeling for Distant Multi-talker Speech Recognition with Single- and Multi-channel Branches

By Naoyuki Kanda, Yusuke Fujita, Shota Horiguchi, Rintaro Ikeshita, Kenji Nagamatsu, Shinji Watanabe. 2019

45 citations recorded by the source corpus.

This paper presents a novel heterogeneous-input multi-channel acoustic model (AM) that has both single-channel and multi-channel input branches. In our proposed training pipeline, a single-channel AM is trained first, then a multi-channel AM is trained starting from the single-channel AM with a randomly initialized multi-channel input branch. Our model uniquely uses the power of a complemen-tal speech enhancement (SE) module while exploiting the power of jointly trained AM and SE architecture. Our method was the foundation for the Hitachi/JHU CHiME-5 system that achieved the second-best result in the CHiME-5 competition, and this paper details various investigation results that we were not able to present during the competition period. We also evaluated and reconfirmed our method’s effectiveness with the AMI Meeting Corpus. Our AM achieved a 30.12% word error rate (WER) for the development set and a 32.33% WER for the evaluation set for the AMI Corpus, both of which are the best results ever reported to the best of our knowledge.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Neural Speaker Diarization with Speaker-Wise Chain Rule (2020, background): “Recent studies on the home-party scenario (CHiME-5 [10]) reported that speaker diarization helped improve automatic speech recognition performance [11–13].”
Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone (2021, methodology): “While promising results were shown, all the previous studies were limited to either simulated data [16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27] or small-scale real data [7, 9, 8, 10](1).”
End-to-End Neural Speaker Diarization with Permutation-Free Objectives (2019, background): “It also helps automatic speech recognition performance in multi-speaker conversation scenarios in meetings (ICSI [3,4], AMI [5,6]) and home environments (CHiME-5 [6–10]).”
A Review of Speaker Diarization: Recent Advances with Deep Learning (2021, methodology): “VB-HMM [53], VBx [54] Out of scope Joint front-end & ASR [55, 56, 57, 58, 59, 60], joint speaker identification & speech separation [61, 62], etc.”