Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis

By Zhizheng Wu, Cassia Valentini-Botinhao, Oliver Watts, Simon King. 2015

270 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Frank K. Soong
2. Junichi Yamagishi
3. Xu Tan
4. Dacheng Tao
5. Tao Qin
6. Lei Xie
7. Tie-Yan Liu
8. Zhizheng Wu

In Signal Processing

1. Frank K. Soong
2. Zhizheng Wu
3. Lei Xie
4. Simon King
5. Haizhou Li
6. Shan Yang
7. Oliver Watts
8. Lei He

Citations in the corpus

• An Overview of Multi-task Learning (2018, neutral): “Applications of MTL in speech include speech synthesis [114,115] and those for natural language processing include joint learning of six NLP tasks (i.e. part-of-speech tagging, chunking, named entity recognition, semantic role labeling, language modeling and semantically related words) [116],…”
• A Survey on Multi-Task Learning (2017, background): “In [195], a multi-task stacked deep neural network, which consists”
• GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks (2017, background): “…other fields, from natural language processing (Hashimoto et al. (2016); Collobert & Weston (2008); Søgaard & Goldberg (2016)) to speech synthesis (Wu et al. (2015); Seltzer & Droppo (2013)), from very domain-specific applications like traffic prediction (Huang et al. (2014)) to very general…”
• Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering (2017, background): “…and Weston, 2008; Dong et al., 2015; Hashimoto et al., 2016; Liu et al., 2015a; Luong et al., 2016), speech (Huang et al., 2013; 2015; Seltzer and Droppo, 2013; Wu et al., 2015), and reinforcement learning (Devin et al., 2016; Fernando et al., 2017; Jaderberg et al., 2017; Rusu et al., 2016).”
• Merlin: An Open Source Neural Network Speech Synthesis System (2016, methodology): “Deep feedforward neural networks (DNNs) as a deep conditional model are the model popular model in the literature to map linguistic features to acoustic features directly [17, 18, 19, 20, 21].”
• Speaker and language factorization in DNN-based TTS synthesis (2016, background): “Deep Neural Networks (DNNs) have advanced parametric Text-to-Speech (TTS) synthesis to a new frontier [2, 3, 4, 5, 6, 7, 8, 9, 10].”
• Investigating gated recurrent neural networks for speech synthesis (2016, unknown): “To include temporal constraints, we proposed to include contextual information by stacking low-dimensional bottleneck features from multiple consecutive frames [12].”
• A DNN-based Mandarin-Tibetan cross-lingual speech synthesis (2018, methodology): “[11] achieved DNN-based speech synthesis on the basis of the HMM speech synthesis framework to set up the speech spectrum conversion model and the experiment proved that the method can effectively improve the synthesized speech quality.”
• Statistical parametric speech synthesis using generative adversarial networks under a multi-task learning framework (2017, methodology): “More specifically, inspired by [16, 12, 22, 23], we combine the MSE loss with the GAN loss under an MTL framework.”
• Siri On-Device Deep Learning-Guided Unit Selection Text-to-Speech System (2017, methodology): “Although only parts of outputs from MDN and RMDN are used for cost calculation in a later step, all other features are still trained due to the benefit of multitask learning [14].”
• Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS (2020, methodology): “It has also been used in speech synthesis [19], such as statistical parametric speech synthesis with GANs [20] and DNN-based speech synthesis with stacked bottleneck features [21].”
• Median-based generation of synthetic speech durations using a non-parametric approach (2016, unknown): “Steady improvements have been made in statistical parametric speech synthesis (SPSS), in particular through the adoption of deep and recurrent machine-learning techniques in recent years [1, 2].”
• Amalgamating Knowledge from Heterogeneous Graph Neural Networks (2021, background): “In the past few years, multi-task learning has been widely studied in various areas, such as bioinformatics [10, 20], ubiquitous computing [39, 40], natural language processing [4, 38] and computer vision [21, 54, 15, 14].”