Multi-talker ASR for an unknown number of sources: Joint training of source counting, separation and ASR
By Thilo von Neumann, Christoph Boeddeker, Lukas Drude, Keisuke Kinoshita, Marc Delcroix, Tomohiro Nakatani, Reinhold Haeb-Umbach. 2020
46 citations recorded by the source corpus.
Most approaches to multi-talker overlapped speech separation and recognition assume that the number of simultaneously active speakers is given, but in realistic situations, it is typically unknown. To cope with this, we extend an iterative speech extraction system with mechanisms to count the number of sources and combine it with a single-talker speech recognizer to form the first end-to-end multi-talker automatic speech recognition system for an unknown number of active speakers. Our experiments show very promising performance in counting accuracy, source separation and speech recognition on simulated clean mixtures from WSJ0-2mix and WSJ0-3mix. Among others, we set a new state-of-the-art word error rate on the WSJ0-2mix database. Furthermore, our system generalizes well to a larger number of speakers than it ever saw during training, as shown in experiments with the WSJ0-4mix database.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Signal Processing
Citations in the corpus
- Single Channel Voice Separation for Unknown Number of Speakers Under Reverberant and Noisy Settings (2020, background): “The authors of [20] proposed a similar one speaker at a time solution however they were mainly interested in automatic speech recognition as the”
- Investigation of Practical Aspects of Single Channel Speech Separation for ASR (2021, methodology): “The joint optimization of the front-end with the back-end ASR was also shown to be effective in both speech enhancement [22, 23, 24] and separation [25, 26].”