Joint Phoneme Alignment and Text-Informed Speech Separation on Highly Corrupted Speech
By Kilian Schulze-Forster, Clement S. J. Doire, Gaël Richard, Roland Badeau. 2020
21 citations recorded by the source corpus.
Speech separation quality can be improved by exploiting textual information. However, this usually requires text-to-speech alignment at phoneme level. Classical alignment methods are made for rather clean speech and do not work as well on corrupted speech. We propose to perform text-informed speech-music separation and phoneme alignment jointly using recurrent neural networks and the attention mechanism. We show that it leads to benefits for both tasks. In experiments, phoneme transcripts are used to improve the perceived quality of separated speech over a non-informed baseline. Moreover, our novel phoneme alignment method based on the attention mechanism achieves state-of-the-art alignment accuracy on clean and on heavily corrupted speech.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jinyu Li
- Zhuo Chen
- Takuya Yoshioka
- Shujie Liu
- Sanyuan Chen
- Naoyuki Kanda
- Xiaofei Wang
- Sefik Emre Eskimez
In Signal Processing
- Jinyu Li
- Zhuo Chen
- Takuya Yoshioka
- Shujie Liu
- Sanyuan Chen
- Naoyuki Kanda
- Xiaofei Wang
- Shinji Watanabe
Citations in the corpus
- SpeechX: Neural Codec Language Model as a Versatile Speech Transformer (2023, background): “Furthermore, there has been a lack of comprehensive audio-text-based speech enhancement models that leverage reference transcriptions to generate intelligible speech, except for limited studies focusing only on particular speech enhancement tasks [27], [28].”