SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speech

By Zhenhui Ye, Zhou Zhao, Yi Ren, Fei Wu. 2022

31 citations recorded by the source corpus.

The recent progress in non-autoregressive text-to-speech (NAR-TTS) has made fast and high-quality speech synthesis possible. However, current NAR-TTS models usually use phoneme sequence as input and thus cannot understand the tree-structured syntactic information of the input sequence, which hurts the prosody modeling. To this end, we propose SyntaSpeech, a syntax-aware and light-weight NAR-TTS model, which integrates tree-structured syntactic information into the prosody modeling modules in PortaSpeech. Specifically, 1) We build a syntactic graph based on the dependency tree of the input sentence, then process the text encoding with a syntactic graph encoder to extract the syntactic information. 2) We incorporate the extracted syntactic encoding with PortaSpeech to improve the prosody prediction. 3) We introduce a multi-length discriminator to replace the flow-based post-net in PortaSpeech, which simplifies the training pipeline and improves the inference speed, while keeping the naturalness of the generated audio. Experiments on three datasets not only show that the tree-structured syntactic information grants SyntaSpeech the ability to synthesize better audio with expressive prosody, but also demonstrate the generalization ability of SyntaSpeech to adapt to multiple languages and multi-speaker text-to-speech. Ablation studies demonstrate the necessity of each component in SyntaSpeech. Source code and audio samples are available at https://syntaspeech.github.io.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Rongjie Huang
2. Ziyue Jiang
3. Zhou Zhao
4. Yi Ren
5. Jinglin Liu
6. Zhe Ye
7. Huadai Liu
8. Chen Zhang

Citations in the corpus

• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias (2023, methodology): “The discriminator follows the architecture proposed in SyntaSpeech [63].”
• TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation (2022, background): “…works bring NAR model into other sequence-to-sequence tasks (Cui et al., 2021; Ye et al., 2023; Huang et al., 2022c; Yang et al., 2022b), such as video caption (Yang et al., 2019), speech recognition (Chen et al., 2020) and speech synthesis (Ye et al., 2022; Huang et al., 2022d; Yang et al., 2023).”