A Review of Deep Learning Based Speech Synthesis

By Yishuang Ning, Sheng He, Zhiyong Wu, Chunxiao Xing, Liang-Jie Zhang. 2019

164 citations recorded by the source corpus.

Speech synthesis, also known as text-to-speech (TTS), has attracted increasingly more attention. Recent advances on speech synthesis are overwhelmingly contributed by deep learning or even end-to-end techniques which have been utilized to enhance a wide range of application scenarios such as intelligent speech interaction, chatbot or conversational artificial intelligence (AI). For speech synthesis, deep learning based techniques can leverage a large scale of pairs to learn effective feature representations to bridge the gap between text and speech, thus better characterizing the properties of events. To better understand the research dynamics in the speech synthesis field, this paper firstly introduces the traditional speech synthesis methods and highlights the importance of the acoustic modeling from the composition of the statistical parametric speech synthesis (SPSS) system. It then gives an overview of the advances on deep learning based speech synthesis, including the end-to-end approaches which have achieved start-of-the-art performance in recent years. Finally, it discusses the problems of the deep learning methods for speech synthesis, and also points out some appealing research directions that can bring the speech synthesis research into a new frontier.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Cong Han
5. Nima Mesgarani
6. Frank K. Soong
7. Gavin Mischler
8. Vinay S. Raghavan

In Signal Processing

1. Xu Tan
2. Tie-Yan Liu
3. Tao Qin
4. Cong Han
5. Nima Mesgarani
6. Frank K. Soong
7. Zhaoxi Mu
8. Xinyu Yang

Citations in the corpus

• StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models (2023, background): “Text-to-speech (TTS) synthesis has seen significant advancements in recent years, with numerous applications such as virtual assistants, audiobooks, and voice-over narration benefiting from increasingly natural and expressive synthetic speech [1, 2].”
• A Survey on Neural Speech Synthesis (2021, background): “While there are multiple survey papers on statistical parametric speech synthesis [27, 418, 351, 415] and neural TTS [325, 220, 300, 242, 115, 254, 236], a comprehensive survey on the basics and the recent developments of neural TTS is still necessary since the topics in this area are diverse and ev”
• A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, unknown): “Other works [73, 74, 112] review speech synthesis based on deep learning, which give us an understanding of the development of neural text-to-speech or speech enhancement.”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “[148] and Sruthi and Meharban [197] reviewed TTS methods based on deep learning.”
• Hierarchical Transfer Learning for Multilingual, Multi-Speaker, and Style Transfer DNN-Based TTS on Low-Resource Languages (2020, background): “In general, modern TTS involves three main processes: text analysis, acoustic modeling, and synthesizing speech waveforms [4].”