Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features

By Siddharth Gururani, Kilol Gupta, Dhaval Shah, Zahra Shakeri, Jervis Pinto. 2019

19 citations recorded by the source corpus.

This paper presents a simple yet effective method to achieve prosody transfer from a reference speech signal to synthesized speech. The main idea is to incorporate well-known acoustic correlates of prosody such as pitch and loudness contours of the reference speech into a modern neural text-to-speech (TTS) synthesizer such as Tacotron2 (TC2). More specifically, a small set of acoustic features are extracted from reference audio and then used to condition a TC2 synthesizer. The trained model is evaluated using subjective listening tests and a novel objective evaluation of prosody transfer is proposed. Listening tests show that the synthesized speech is rated as highly natural and that prosody is successfully transferred from the reference speech signal to the synthesized signal.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhaoxi Mu
2. Xinyu Yang
3. Yizhuo Dong

In Signal Processing

1. Zhaoxi Mu
2. Xinyu Yang
3. Yizhuo Dong

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, neutral): “Expressive Modeling Variation Information GST-Tacotron [377], Ref-Tacotron [303], [39] Multi-SpectroGAN [182], PriorGrad [181], [48] VAE-TTS [436], Glow-TTS [156], [116, 117] [99, 4, 360, 100, 318, 319, 360, 68, 138]”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “[69] refined the model on the basis of Skerry-Ryan et al.”