TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model

By Stanislav Beliaev, Yurii Rebryk, Boris Ginsburg. 2020

20 citations recorded by the source corpus.

We propose TalkNet, a convolutional non-autoregressive neural model for speech synthesis. The model consists of two feed-forward convolutional networks. The first network predicts grapheme durations. An input text is expanded by repeating each symbol according to the predicted duration. The second network generates a mel-spectrogram from the expanded text. To train a grapheme duration predictor, we add the grapheme duration to the training dataset using a pre-trained Connectionist Temporal Classification (CTC)-based speech recognition model. The explicit duration prediction eliminates word skipping and repeating. Experiments on the LJSpeech dataset show that the speech quality nearly matches auto-regressive models. The model is very compact -- it has 10.8M parameters, almost 3x less than the present state-of-the-art text-to-speech models. The non-autoregressive architecture allows for fast training and inference.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Ye Jia
6. Yu Zhang
7. Heiga Zen
8. Yonghui Wu

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Replacing Attention with Duration FastSpeech [284], DurIAN [411], EATS [67] Glow-TTS [156], [190, 19, 355, 422, 73, 298]”
• Parallel Tacotron: Non-Autoregressive and Controllable TTS (2020, background): “Some of these models train their explicit duration model to predict groundtruth phoneme duration extracted from autoregressive models [15, 18, 22], internal aligner models [17, 19], or external aligner models [16, 23, 24].”
• Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, methodology): “On the contrary, NAR models [12-21] achieve extremely fast inference speed, where a length regulator is used to replace the attention module in AR models to predict phoneme durations and expand the phoneme embeddings for parallel generation.”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “For example, TalkNet [12] uses the CTC-based automatic speech recognition (ASR) model Quartznet [111], FastSpeech 2 [174] uses the forcedalignment tool MFA toolkit [135], DurIAN [243] uses a external alignment model [51, 252], RobuTrans [119] uses speech recognition tools, Parallel Tacotron [49] and”
• Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling (2020, methodology): “TalkNet (Beliaev et al., 2020) uses a CTC-based ASR model to extract target durations, while CHiVE (Kenter et al., 2019), FastSpeech 2 (Ren et al., 2020), and DurIAN (Yu et al., 2019) use target durations from an external aligner module utilizing forced alignment.”
• Hi-Fi Multi-Speaker English TTS Dataset (2021, background): “These two datasets are frequently used for single-speaker TTS research since they provide a relatively large number of hours per speaker [7, 8, 9, 10, 11].”