AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines

By Yao Shi, Hui Bu, Xin Xu, Shaojing Zhang, Ming Li. 2020

289 citations recorded by the source corpus.

In this paper, we present AISHELL-3, a large-scale and high-fidelity multi-speaker Mandarin speech corpus which could be used to train multi-speaker Text-to-Speech (TTS) systems. The corpus contains roughly 85 hours of emotion-neutral recordings spoken by 218 native Chinese mandarin speakers. Their auxiliary attributes such as gender, age group and native accents are explicitly marked and provided in the corpus. Accordingly, transcripts in Chinese character-level and pinyin-level are provided along with the recordings. We present a baseline system that uses AISHELL-3 for multi-speaker Madarin speech synthesis. The multi-speaker speech synthesis system is an extension on Tacotron-2 where a speaker verification model and a corresponding loss regarding voice similarity are incorporated as the feedback constraint. We aim to use the presented corpus to build a robust synthesis model that is able to achieve zero-shot voice cloning. The system trained on this dataset also generalizes well on speakers that are never seen in the training process. Objective evaluation results from our experiments show that the proposed multi-speaker synthesis system achieves high voice similarity concerning both speaker embedding similarity and equal error rate measurement. The dataset, baseline system code and generated samples are available online.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Xu Tan
3. Tie-Yan Liu
4. Zhou Zhao
5. Yi Ren
6. Jinglin Liu
7. Frank K. Soong
8. Rongjie Huang

Citations in the corpus

• UniAudio: An Audio Foundation Model Toward Universal Audio Generation (2023, methodology): “For the text-to-music task, we follow MusicGen Copet et al. (2023), evaluating our methods on MusicCaps Agostinelli et al. (2023).”
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias (2023, methodology): “To evaluate the zero-shot performance of Mega-TTS, we perform experiments on VCTK [57], AISHELL-3 [51] and LibriSpeech test-clean [42] datasets.”
• A Survey on Neural Speech Synthesis (2021, background): “Other works [238, 202, 30, 299, 393, 6] leverage the feedback loss to ensure the controllability on style and speaker embeddings, etc.”
• InstructTTS: Modelling Expressive TTS in Discrete Latent Space With Natural Language Style Prompt (2023, methodology): “In our study, we concatenate all of the tokens from the first codebook to the last codebook.”
• Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus (2021, unknown): “AISHELL-3 [36] contains roughly 85 hours of emotion-neutral recordings spoken by 218 native Chinese speakers, which could be applied for multi-speaker speech synthesis.”
• SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models (2023, methodology): “For Chinese, we select samples from the Aishell-3 dataset (Shi et al., 2021) for testing.”
• Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts (2023, unknown): “GigaSpeech (Chen et al., 2021a) 8.1M 396.9M 9.6k LibriLight (Kahn et al., 2020) 10.0M 652.3M 17.9k LibriTTS (Zen et al., 2019) 375.1k 23.6M 573.2 WenetSpeech (Zhang et al., 2022) 14.0M 325.5M 9.3K AISHELL-3 (Shi et al., 2020) 88.0k 2.2M 85.0 Didi 500 (Guo et al., 2021) 52 prosodic prompt u rhy and u”
• Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis (2022, methodology): “The end-to-end TTS technology allows to train a model only with audio files and paired textual transcriptions, such as corpora LJ-Speech [25], VCTK [26], Aishell-3 [27], and DiDiSpeech [28].”
• Review of end-to-end speech synthesis technology based on deep learning (2021, background): “AISHELL-3(7) [191] is a high-quality Mandarin corpus for multi-speaker TTS published by Shell Shell.”
• Effective and Differentiated Use of Control Information for Multi-speaker Speech Synthesis (2021, methodology): “The data set of our experiments is the public multi-speaker mandarin corpus AISHELL-3 [28], which contains roughly 85 hours of recordings spoken by 218 native Chinese mandarin speakers.”
• EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusion (2021, background): “Four speech editing systems are developed on four datasets [23–25] respectively.”