DurIAN: Duration Informed Attention Network for Speech Synthesis

By Chengzhu Yu, Heng Lu, Na Hu, Meng Yu, Chao Weng, Kun Xu, Peng Liu, Deyi Tuo, Shiyin Kang, Guangzhi Lei and 2 more. 2020

80 citations recorded by the source corpus.

In this paper, we present a robust and effective speech synthesis system that generates highly natural speech. The key component of proposed system is Duration Informed Attention Network (DurIAN), an autoregressive model in which the alignments between the input text and the output acoustic features are inferred from a duration model. This is different from the attention mechanism used in existing end-to-end speech synthesis systems that accounts for various unavoidable artifacts. To improve the audio generation efﬁciency of neural vocoders, we also propose a multi-band audio generation framework exploiting the sparseness characteristics of neural network. With proposed multi-band processing framework, the total computational complexity of WaveRNN model can be effectively reduced from 9.8 to 3.6 GFLOPS without any performance loss. Finally, we show that proposed DurIAN system could generate highly natural speech that is on par with current state of the art end-to-end systems, while being robust and stable at the same time.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

A Survey on Neural Speech Synthesis (2021, methodology): “Replacing Attention with Duration FastSpeech [284], DurIAN [411], EATS [67] Glow-TTS [156], [190, 19, 355, 422, 73, 298]”
Controllable Emphasis with zero data for text-to-speech (2023, result): “It is similar to D URIAN + from [21], which is inspired by D URIAN [22] and F ASTSPEECH [23].”
CopyCat2: A Single Model for Multi-Speaker TTS and Many-to-Many Fine-Grained Prosody Transfer (2022, background): “[27] C. Yu, H. Lu, N. Hu, M. Yu, C. Weng, K. Xu, P. Liu, D. Tuo, S. Kang, G. Lei et al., “DurIAN: Duration Informed Attention Network for Speech Synthesis,” in Proc. of Interspeech, 2020.”
The Multi-Speaker Multi-Style Voice Cloning Challenge 2021 (2021, unknown): “While recently proposed end-to-end speech synthesis systems, e.g., Tacotron [1], DurIAN [2] and FastSpeech [3], are able to generate high-quality and natural sounding speech, these models usually rely on a large amount of training data from a single speaker.”
Avocodo: Generative Adversarial Network for Artifact-free Vocoder (2022, background): “PQMF, a digital ﬁlter, satisﬁes this requirement (Yu et al. 2020; Yang et al. 2021).”
A Streamwise Gan Vocoder for Wideband Speech Coding at Very Low Bit Rate (2021, unknown): “5 GMACs Multi-band WaveRNN [19] 2 .”
Triple M: A Practical Text-to-Speech Synthesis System with Multi-Guidance Attention and Multi-Band Multi-Time LPCNet (2021, methodology): “Durian proposed in [18] introduces multi-band processing into a single WaveRNN model, which reduces the total computational complexity from 9.8 to 3.6 GFLOPS.”
Any-to-Many Voice Conversion With Location-Relative Sequence-to-Sequence Modeling (2020, methodology): “The DurIAN synthesis model used in this paper is inspired by [32], which is trained to predict the mel-spectrogram X from”
JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech (2022, methodology): “An acoustic feature generator can be autoregressive and attention-based for implicit speech-text alignments [1], [2] or it can be non-autoregressive for efﬁcient parallel inference and duration informed for robustness on synthesis error [3], [4], [5].”