VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop

By Yaniv Taigman, Lior Wolf, Adam Polyak, Eliya Nachmani. 2017

158 citations recorded by the source corpus.

Subfields: Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Junichi Yamagishi
2. Lei He
3. Xu Tan
4. Tao Qin
5. Tie-Yan Liu
6. Wei Ping
7. Hung-yi Lee
8. Xin Eric Wang

In Speech & Audio

1. Xu Tan
2. Tao Qin
3. Lei He
4. Tie-Yan Liu
5. Sheng Zhao
6. Junichi Yamagishi
7. Yu Zhang
8. Hung-yi Lee

Citations in the corpus

• Jukebox: A Generative Model for Music (2020, neutral): “By learning and manipulating auxiliary embeddings, models can mimic a new voice (Arık et al., 2018a; Taigman et al., 2018) at test time.”
• DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, background): “, 2017), VoiceLoop (Taigman et al., 2018), Parallel WaveNet (van den Oord et al.”
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (2018, methodology): “In contrast, VoiceLoop [18] proposed a novel architecture based on a fixed size memory buffer which can generate speech from voices unseen during training.”
• Neural Speech Synthesis with Transformer Network (2018, methodology): “Moreover, VoiceLoop (Taig-man et al. 2018) is an alternative neural TTS method mimicking a person’s voice based on samples captured in-the-wild, such as audios of public speeches, and even with an inaccurate automatic transcripts.”
• Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech (2021, background): “There are two general approaches to deal with such task [8], speaker adaptation [8], [9], [10], [11], [12] and speaker encoding [6], [13], [7], [14], [15], [16], [17].”
• A Survey on Neural Speech Synthesis (2021, methodology): “VoiceLoop [327] adopts a working memory called phonological loop to generate acoustic features (e.”
• Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech (2021, background): “Autoregressive TTS models (Taigman et al., 2018; Shen et al., 2018; Valle et al., 2021) generate diverse speech with different rhythms through their autoregressive structure and several tricks including maintaining dropout probability during inference and priming (Graves, 2013).”
• Neural Voice Cloning with a Few Samples (2018, background): “Fine-tuning can be applied to either the speaker embedding [Taigman et al., 2018] or the whole model.”
• Fitting New Speakers Based on a Short Untranscribed Sample (2018, methodology): “The various parameters follow the implementation released by Taigman et al. (2018).”
• Deep Voice 3: 2000-Speaker Neural Text-to-Speech (2017, background): “…the problem of synthesizing speech with neural networks, including Deep Voice 1 (Arık et al., 2017), Deep Voice 2 (Arık et al., 2017), Tacotron (Wang et al., 2017), Char2Wav (Sotelo et al., 2017), VoiceLoop (Taigman et al., 2017), SampleRNN (Mehri et al., 2017), and WaveNet (Oord et al., 2016).”
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech (2018, methodology): “Several neural TTS systems were proposed, including Deep Voice 1 (Arık et al., 2017a), Deep Voice 2 (Arık et al., 2017b), Deep Voice 3 (Ping et al., 2018), Tacotron (Wang et al., 2017), Tacotron 2 (Shen et al., 2018), Char2Wav (Sotelo et al., 2017), and VoiceLoop (Taigman et al., 2018).”
• Hierarchical Generative Modeling for Controllable Speech Synthesis (2018, background): “(2018) extend VoiceLoop (Taigman et al., 2018) with a latent reference embedding generated by a VAE, using a centered fixed-variance isotropic Gaussian prior for the latent attributes.”
• End-to-End Adversarial Text-to-Speech (2020, background): “, 2020), and VoiceLoop (Taigman et al., 2017) are autoregressive models that generate spectrograms or vocoder features frame by frame.”
• WaveFlow: A Compact Flow-based Model for Raw Audio (2019, neutral): “, 2017), VoiceLoop (Taigman et al., 2018), WaveRNN (Kalchbrenner et al.”
• Adaspeech 2: Adaptive Text to Speech with Untranscribed Data (2021, background): “There indeed exist some works [11, 12, 13] using untranscribed data for TTS adaptation, which requires joint training of TTS pipeline and the modules used for adaptation.”
• Wasserstein GAN and Waveform Loss-Based Acoustic Model Training for Multi-Speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder (2018, methodology): “VoiceLoop [26] involves the data of 109 speakers for acoustic model training, and Deep Voice 3 [27] trained a multi-speaker model using over 2,000 speakers.”