Merlin: An Open Source Neural Network Speech Synthesis System

By Zhizheng Wu, Oliver Watts, Simon King. 2016

345 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Junichi Yamagishi
5. Wei-Ning Hsu
6. Hung-yi Lee
7. James R. Glass
8. Frank K. Soong

In Speech & Audio

1. Wei-Ning Hsu
2. Xu Tan
3. Junichi Yamagishi
4. Tao Qin
5. Hung-yi Lee
6. Tie-Yan Liu
7. Yossi Adi
8. Sheng Zhao

Citations in the corpus

• Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, methodology): “The ZRSC’19 baseline system uses a pipeline of a Dirichlet process Gaussian mixture model (DPGMM) for acoustic unit discovery [13], and the Merlin neural network speech synthe-siser [45] trained on the discovered units from the target speaker.”
• The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems (2018, methodology): “The voice transformation system is based on the Merlin Toolkit [24].”
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech (2020, methodology): “Compared with parametric speech synthesis systems such as Merlin [26] and Deep Voice [1], Fastspeech 2 and 2s employ the features such as duration and pitch fully end-to-end, and adopt self-attention based feed-forward network to generate mel-spectrograms or waveform in parallel.”
• The Zero Resource Speech Challenge 2019: TTS without T (2019, methodology): “This baseline uses out-of-the-box acoustic unit discovery [5] and an out-of-the-box speech synthesizer (Merlin, with the Ossian front end [6]).”
• Voice Synthesis for in-the-Wild Speakers via a Phonological Loop (2017, methodology): “Similar to [18], these features were computed using the Merlin toolkit [23].”
• Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech (2022, background): “In order to handle this problem, some works have tried to enrich the phoneme sequence input with syntactic information and linguistic features [11–13].”
• VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop (2017, methodology): “Similar to (Sotelo et al., 2017), these features were computed using the Merlin toolkit (Wu et al., 2016).”
• fairseq Sˆ2: A Scalable and Integrable Speech Synthesis Toolkit (2021, methodology): “, 2007), Merlin (Wu et al., 2016), STRAIGHT (Kawahara et al.”
• ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech (2019, methodology): “The alignment for model training was extracted using the Merlin toolkit [21].”
• High-Quality Nonparallel Voice Conversion Based on Cycle-Consistent Adversarial Network (2018, methodology): “We compared the proposed method with two state-of-the-art parallel VC methods, one based on a Merlin system and the other based on a GAN.”
• An Investigation of Subband Wavenet Vocoder Covering Entire Audible Frequency Range with Limited Acoustic Features (2018, methodology): “…SPSS and VC systems, we introduced lower-dimensional acoustic features constructed from the fundamental frequency and simple mel-cepstral coefficients rather than the mel-spectrograms used in Deep Voice 3 [16] and Tacotron 2 [17] and higher-dimensional STRAIGHT-and WORLD-based features [13, 22–24].”
• Virtual Phone Discovery for Speech Synthesis Without Text (2019, methodology): “Merlin [23] open source toolkit is used in this work to build a neural network based TTS system.”
• Unsupervised Acoustic Unit Representation Learning for Voice Conversion using WaveNet Auto-encoders (2020, methodology): “The challenge official baseline method is a combination of a DPGMM [15] system and a Merlin [38] system.”
• Text-to-Speech Synthesis Using Found Data for Low-Resource Languages (2019, methodology): “71 6.7 Percent preference for Merlin AVM adapted to subsets of the data selected based on high, middle, or low values for various acoustic and prosodic features. . . . . . . . . 72 6.8 Percent preference for Merlin voices trained on data labeled as having high, medium, or low values for features and”
• Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems (2019, methodology): “To compare the seq2seq model with full-context label input using a conventional DNN-based AM, a simple bidirectional LSTM-based AM [3,5] is introduced, as shown in Fig 2(b).”
• Sequence to Sequence Neural Speech Synthesis with Prosody Modification Capabilities (2019, background): “Alternative neural TTS systems [5][6] predict the speech acoustics from the symbolic sequence augmented with detailed prosodic information (including at least phone durations and pitch trajectory).”