A Chapter-Wise Understanding System for Text-To-Speech in Chinese Novels
By Junjie Pan, Lin Wu, Xiang Yin, Pengfei Wu, Chenchang Xu, Zejun Ma. 2021
15 citations recorded by the source corpus.
In TTS-based audiobook production, multi-role dubbing and emotional expressions can significantly improve the naturalness of audiobooks. However, it requires manual annotation of original novels with explicit speaker and emotion tags in sentence level, which is extremely time-consuming and costly. In this paper, we propose a chapter-wise understanding system for Chinese novels, to predict speaker and emotion tags automatically based on the chapter-level context. Compared with baselines of each component, our models obtain higher performance. Audiobooks produced by our proposed system along with a multi-speaker emotional TTS system, are proved to achieve comparable quality score to audiobooks made by individual producers. Demos are demonstrated in https://jeffpan.net/icassp/2021/main.html.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shinnosuke Takamichi
- Hiroshi Saruwatari
- Ashish Kulkarni
- Sri Karlapati
- Detai Xin
- Alexis Moinet
- Thomas Drugman
- Bartosz Putrycz
In Signal Processing
- Shinnosuke Takamichi
- Hiroshi Saruwatari
- Detai Xin
- Alexis Moinet
- Thomas Drugman
- Bartosz Putrycz
- Bastian Schnell
- Sharath Adavanne
Citations in the corpus
- BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (2024, background): “Cars displayed in these showrooms with ENERGY-EFFICIENT AND GREEN decals prominently featured across the windshield aren’t announcing environmentalism; they’re virtue signaling.”
- eCat: An End-to-End Model for Multi-Speaker TTS & Many-to-Many Fine-Grained Prosody Transfer (2023, background): “On a parallel note, there has also been work in multi-sentence TTS or long-context TTS [13–16].”
- Improving Speech Prosody of Audiobook Text-To-Speech Synthesis with Acoustic and Textual Contexts (2022, background): “Single-sentence methods synthesize one utterance at a time, and improves speech prosody by feeding auxiliary features like context [8, 9, 10], emotions [13], and features extracted by DNNs like variational au-toencoder [14, 15] and global style tokens (GST) [16].”