XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System

By Peiling Lu, Jie Wu, Jian Luan, Xu Tan, Li Zhou. 2020

117 citations recorded by the source corpus.

This paper presents XiaoiceSing, a high-quality singing voice synthesis system which employs an integrated network for spectrum, F0 and duration modeling. We follow the main architecture of FastSpeech while proposing some singing-specific design: 1) Besides phoneme ID and position encoding, features from musical score (e.g.note pitch and length) are also added. 2) To attenuate off-key issues, we add a residual connection in F0 prediction. 3) In addition to the duration loss of each phoneme, the duration of all the phonemes in a musical note is accumulated to calculate the syllable duration loss for rhythm enhancement. Experiment results show that XiaoiceSing outperforms the baseline system of convolutional neural networks by 1.44 MOS on sound quality, 1.18 on pronunciation accuracy and 1.38 on naturalness respectively. In two A/B tests, the proposed F0 and duration modeling methods achieve 97.3% and 84.3% preference rate over baseline respectively, which demonstrates the overwhelming advantages of XiaoiceSing.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Yi Ren
5. Zhou Zhao
6. Jinglin Liu
7. Frank K. Soong
8. Sheng Zhao

In Signal Processing

1. Xu Tan
2. Yi Ren
3. Zhou Zhao
4. Tao Qin
5. Tie-Yan Liu
6. Jinglin Liu
7. Frank K. Soong
8. Feiyang Chen

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Due to page limitations, we only reviewed core algorithms of TTS; readers can refer to other papers for TTS related problems and applications, such as voice conversion [302], singing voice synthesis [112, 211, 35], talking face synthesis [36], etc.”
• DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (2021, background): “Eighteen qualified listeners are asked to make judgments about the synthesized song samples in terms of three aspects: pronunciation accuracy, sound quality and naturalness [16].”
• M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus (2022, unknown): “…of deep learning and the continuous breakthroughs in TTS techniques [47; 23; 52; 37; 35; 36; 18], singing voice synthesis [45; 21; 14; 5; 38; 2; 27; 40; 9; 13; 10; 51; 17; 8; 43; 1; 22; 48; 54; 41; 19; 56; 53; 25] and its related tasks (e.g., SVC [33; 42; 24; 46; 55]) have received extensive…”
• CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model (2023, background): “, 2019] [Lu et al., 2020], in which the acoustic model first converts the textual and other controlling information into acoustic features (e.”
• Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis (2022, background): “While several open-source SVS databases designed for English [29, 30] or other languages [31] have been released and also lots of research has been conducted towards Mandarin singing voice synthesis [24, 32, 33], no accessible public Mandarin SVS corpus with high quality exists, thereby limiting the”
• Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus (2021, unknown): “Following the essential components similar to TTS systems, SVS systems generally adopt an acoustic model [5, 23] to convert the musical scores into acoustic features, and a vocoder [2, 21] to generate audio waveform from acoustic features.”
• A Survey on Recent Deep Learning-driven Singing Voice Synthesis Systems (2021, methodology): “This mechanism utilizes a simple auxiliary decoder seen in Transformer-based acoustic models [2], [6] to generate a rough Mel-spectrogram from the musical score.”
• DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis (2021, methodology): “We compare the MOS of the song samples generated by DiffSinger with the following systems: 1) GT , the ground truth singing audio; 2) GT (Mel + PWG) , where we first convert the ground truth singing audio to the ground truth mel-spectrograms, and then convert these mel-spectrograms back to audio usin”
• HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis (2020, methodology): “We mainly compare HiFiSinger with the following settings and systems: 1) Recording, the original singing recordings; 2) Recording (24kHz), the original singing recordings downsampled to 24kHz; 3) Xiaoic-eSing (Lu et al., 2020), a previous SVS system that also adopts 48kHz sampling rate but leverages”