Expressive Text-to-Speech Using Style Tag

By Minchan Kim, Sung Jun Cheon, Byoung Jin Choi, Jong Jin Kim, Nam Soo Kim. 2021

53 citations recorded by the source corpus.

As recent text-to-speech (TTS) systems have been rapidly improved in speech quality and generation speed, many researchers now focus on a more challenging issue: expressive TTS. To control speaking styles, existing expressive TTS models use categorical style index or reference speech as style input. In this work, we propose StyleTagging-TTS (ST-TTS), a novel expressive TTS model that utilizes a style tag written in natural language. Using a style-tagged TTS dataset and a pre-trained language model, we modeled the relationship between linguistic embedding and speaking style domain, which enables our model to work even with style tags unseen during training. As style tag is written in natural language, it can control speaking style in a more intuitive, interpretable, and scalable way compared with style index or reference speech. In addition, in terms of model architecture, we propose an efficient non-autoregressive (NAR) TTS architecture with single-stage training. The experimental result shows that ST-TTS outperforms the existing expressive TTS model, Tacotron2-GST in speech quality and expressiveness.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wei-Ning Hsu
2. Yossi Adi
3. Xu Tan
4. Tao Qin
5. Tie-Yan Liu
6. Abdel-rahman Mohamed
7. Emmanuel Dupoux
8. Dong Yu

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “Information Type Explicit Language/Style/Speaker ID [438, 241, 191, 158, 38] Pitch/Duration/Energy [284, 286, 177, 155, 233, 359]”
• InstructTTS: Modelling Expressive TTS in Discrete Latent Space With Natural Language Style Prompt (2023, methodology): “The most related to our work are Style-Tagging-TTS (ST-TTS) [34] and PromptTTS [35].”
• Textless Speech Emotion Conversion using Decomposed and Discrete Representations (2021, background): “Although expressive speech synthesis models have shown great success in recent years (Skerry-Ryan et al., 2018a; Ren et al., 2020; Kim et al., 2021; Sorin et al., 2020; Hono et al., 2020), the majority is still text-based, i.e., Text-to-Speech (Tan et al., 2021), and is therefore limited in terms…”
• Self-supervised Context-aware Style Representation for Expressive Speech Synthesis (2022, unknown): “In real applications, the supervised learning is more widely adopted by leveraging explicit labels as auxiliary information to guide multi-style TTS [9, 10].”