GraphTTS: Graph-to-Sequence Modelling in Neural Text-to-Speech

By Aolan Sun, Jianzong Wang, Ning Cheng, Huayi Peng, Zhen Zeng, Jing Xiao. 2020

22 citations recorded by the source corpus.

This paper leverages the graph-to-sequence method in neural text-to-speech (GraphTTS), which maps the graph embedding of the input sequence to spectrograms. The graphical inputs consist of node and edge representations constructed from input texts. The encoding of these graphical inputs incorporates syntax information by a GNN encoder module. Besides, applying the encoder of GraphTTS as a graph auxiliary encoder (GAE) can analyse prosody information from the semantic structure of texts. This can remove the manual selection of reference audios process and makes prosody modelling an end-to-end procedure. Experimental analysis shows that GraphTTS outperforms the state-of-the-art sequence-to-sequence models by 0.24 in Mean Opinion Score (MOS). GAE can adjust the pause, ventilation and tones of synthesised audios automatically. This experimental conclusion may give some inspiration to researchers working on improving speech synthesis prosody.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yiwei Guo
2. Rui Liu
3. Xie Chen
4. Berrak Sisman
5. Chenpeng Du
6. Haizhou Li
7. K. Yu
8. Helen Mei-Ling Meng

Citations in the corpus

• VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature (2022, background): “In addition, [17, 18] introduces syntactic graph and [19, 20] introduces word embeddings to TTS models.”
• Dependency Parsing based Semantic Representation Learning with Graph Neural Network for Enhancing Expressiveness of Text-to-Speech (2021, background): “The gated graph neural network (GGNN) is adopted due to its long-term propagation of information flow [23].”
• SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speech (2022, unknown): “For instance, GraphTTS [Sun et al., 2020] and GraphPB [Sun et al.”
• Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis (2020, background): “Despite the progress [17, 18], Transformer TTS doesn’t explicitly associate input text with output utterances from syntactic point of view at sentence level, which is proven useful in speaking style and prosody modeling [19–22].”