Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

By Rui Liu, Berrak Sisman, Haizhou Li. 2020

27 citations recorded by the source corpus.

Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways. Transformer-based TTS is one of such successful implementations. While Transformer TTS models the speech frame sequence well with a self-attention mechanism, it does not associate input text with output utterances from a syntactic point of view at sentence level. We propose a novel neural TTS model, denoted as GraphSpeech, that is formulated under graph neural network framework. GraphSpeech encodes explicitly the syntactic relation of input lexical tokens in a sentence, and incorporates such information to derive syntactically motivated character embeddings for TTS attention mechanism. Experiments show that GraphSpeech consistently outperforms the Transformer TTS baseline in terms of spectrum and prosody rendering of utterances.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Chenpeng Du
6. K. Yu
7. Yiwei Guo
8. Xinyu Yang

Citations in the corpus

• VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature (2022, background): “In addition, [17, 18] introduces syntactic graph and [19, 20] introduces word embeddings to TTS models.”
• A Survey on Neural Speech Synthesis (2021, background): “Some other ways to incorporate prosody features include 1) reference encoders that learn the prosody representations from reference speech; 2) text pre-training that learns good text representations with implicit prosody information through self-supervised pre-training [101, 95]; and 3) incorporatin”
• Dependency Parsing based Semantic Representation Learning with Graph Neural Network for Enhancing Expressiveness of Text-to-Speech (2021, methodology): “To explicitly incorporate the dependency relations among different lexical tokens in a sentence, GraphSpeech [19] introduces the relations of input lexical tokens to the graph attention mechanism from dependency parsing that is proven to be useful for representing semantic association [20].”
• Expressive TTS Training With Frame and Style Reconstruction Loss (2020, background): “conventional TTS pipeline [8], [9] into an unified framework that learns sequence-to-sequence mapping from text to a sequence of acoustic features [7], [10]–[15].”
• SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speech (2022, unknown): “Later, GraphSpeech [Liu et al., 2021] introduces dependency parsing in the text-to-graph module to better represent the syntactic information of the input sentence, and utilizes bi-directional gated recurrent unit (GRU) to aggregate information through the syntactic graph.”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “In order to improve the quality of synthesized speech, GraphSpeech [126] inputs syntactic knowledge as additional contextual information into the self-attention module of Transformer-TTS [118].”