Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input

By Brooke Stephenson, Thomas Hueber, Laurent Girin, Laurent Besacier. 2021

10 citations recorded by the source corpus.

The prosody of a spoken word is determined by its surrounding context. In incremental text-to-speech synthesis, where the synthesizer produces an output before it has access to the complete input, the full context is often unknown which can result in a loss of naturalness in the synthesized speech. In this paper, we investigate whether the use of predicted future text can attenuate this loss. We compare several test conditions of next future word: (a) unknown (zero-word), (b) language model predicted, (c) randomly predicted and (d) ground-truth. We measure the prosodic features (pitch, energy and duration) and find that predicted text provides significant improvements over a zero-word lookahead, but only slight gains over random-word lookahead. We confirm these results with a perceptive test.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Xinyu Yang
6. Zhaoxi Mu
7. Yizhuo Dong
8. Sheng Zhao

In Signal Processing

1. Xu Tan
2. Tie-Yan Liu
3. Tao Qin
4. Frank K. Soong
5. Zhaoxi Mu
6. Xinyu Yang
7. Yizhuo Dong
8. Sheng Zhao

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “Streaming TTS [74, 217, 316, 399, 317, 231] synthesizes speech once some input tokens are comming, without waiting for the whole input sentence, which can also speed up inference.”
• Direct simultaneous speech to speech translation (2021, unknown): “It incrementally generates utterances word-by-word, utilizing the duration predictor for segmenting output word boundaries (Stephenson et al., 2021).”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “[199] used the language model GPT-2 [169] to predict the next word in the input text, thereby improving the naturalness of speech synthesized by the incremental TTS model by utilizing the predicted contextual information.”