Acoustic Word Embeddings for End-to-End Speech Synthesis

By Feiyu Shen, Chenpeng Du, K. Yu. 2021

4 citations recorded by the source corpus.

The most recent end-to-end speech synthesis systems use phonemes as acoustic input tokens and ignore the information about which word the phonemes come from. However, many words have their specific prosody type, which may significantly affect the naturalness. Prior works have employed pre-trained linguistic word embeddings as TTS system input. However, since linguistic information is not directly relevant to how words are pronounced, TTS quality improvement of these systems is mild. In this paper, we propose a novel and effective way of jointly training acoustic phone and word embeddings for end-to-end TTS systems. Experiments on the LJSpeech dataset show that the acoustic word embeddings dramatically decrease both the training and validation loss in phone-level prosody prediction. Subjective evaluations on naturalness demonstrate that the incorporation of acoustic word embeddings can significantly outperform both pure phone-based system and the TTS system with pre-trained linguistic word embedding.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yiwei Guo
2. Xie Chen
3. Chenpeng Du
4. K. Yu
5. Dong Yu
6. Jinyu Li
7. Kai Yu
8. Li Deng

Citations in the corpus

• VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature (2022, background): “In addition, [17, 18] introduces syntactic graph and [19, 20] introduces word embeddings to TTS models.”