The Zero Resource Speech Challenge 2019: TTS without T

By Ewan Dunbar, Robin Algayres, Julien Karadayi, Mathieu Bernard, Juan Benjumea, Xuan-Nga Cao, Lucie Miskic, Charlotte Dugrain, Lucas Ondel, Alan W. Black and 3 more. 2019

129 citations recorded by the source corpus.

We present the Zero Resource Speech Challenge 2019, which proposes to build a speech synthesizer without any text or phonetic labels: hence, TTS without T (text-to-speech without text). We provide raw audio for a target voice in an unknown language (the Voice dataset), but no alignment, text or labels. Participants must discover subword units in an unsupervised way (using the Unit Discovery dataset) and align them to the voice recordings in a way that works best for the purpose of synthesizing novel utterances from novel speakers, similar to the target speaker's voice. We describe the metrics used for evaluation, a baseline system consisting of unsupervised subword unit discovery plus a standard TTS system, and a topline TTS using gold phoneme transcriptions. We present an overview of the 19 submitted systems from 10 teams and discuss the main results.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Learning Robust and Multilingual Speech Representations (2020, neutral): “Finally, the ZeroSpeech challenges have explicitly looked at correlations between learned representations and phonetic structures that generalize across many languages and adapt to new speakers (Dunbar et al., 2017, 2019).”
Unsupervised Pre-Training of Bidirectional Speech Encoders via Masked Reconstruction (2020, background): “, [14, 15, 16, 17]), where the focus is to learn phonetic or word-like units, or representations that can distinguish among such units.”
Libri-Light: A Benchmark for ASR with Limited or No Supervision (2019, positive): “The representation is evaluated using ABX error, a distance-based metric used in previous zero resource challenges [9, 10, 11].”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, methodology): “The full details of the evaluation can be found in Dunbar et al. (2019).”
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, background): “Learning discrete representations of speech has gathered much recent interest (Versteegh et al., 2016; Dunbar et al., 2019).”
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019 (2019, background): “The source corpora of the surprise language are describe here [21, 22], and further details can be found here [3].”
Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, background): “The test utterance is then re-synthesised in the target speaker’s voice based on these symbols [34].”
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, positive): “Recently, unsupervised learning of low bitrate speech representations was explored under the Zero-Resource Challenge [16, 17].”
On Generative Spoken Language Modeling from Raw Audio (2021, neutral): “We entirely draw on evaluations from the Zero Resource challenge series (Versteegh et al., 2016; Dunbar et al., 2019; Nguyen et al., 2020) 6 for comparability with published work and refer to these challenges for details.”
UWSpeech: Speech to Speech Translation for Unwritten Languages (2020, methodology): “2019) and the more advanced vector quantized VAE (VQ-VAE) (van den Oord, Vinyals et al. 2017; Dunbar et al. 2019; Tjandra et al. 2019; Chorowski et al. 2019; Liu et al. 2019; Tjandra, Sakti, and Nakamura 2019; Baevski, Schneider, and Auli 2019).”
Speech-to-Speech Translation Between Untranscribed Unknown Languages (2019, background): “The Zero Resource Speech Challenge s eries [9, 10, 11] was constructed to progress incrementally toward a system that learns an end-to-end spoken dialog (SD) system in an unknown language from scratch just using information available to language learning infants.”
Cyclic Spectral Modeling for Unsupervised Unit Discovery into Voice Conversion with Excitation and Waveform Modeling (2020, methodology): “The dataset of the 2019-task [10] in ZSC 2020 (1)consisted of English and Surprise [29, 30] languages, where each has a subset of voice dataset and unit dataset.”
The Zero Resource Speech Challenge 2020: Discovering discrete subword and word units (2020, methodology): “roSpeech benchmark [3] (track 1b in Figure 1).”
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units (2020, methodology): “The ABX test evaluates the phone discriminability of the learned units on the ZeroSpeech 2019 English test set (Dunbar et al., 2019).”
Early phonetic learning without phonetic categories: Insights from large-scale simulations on realistic input (2021, background): “Based on these tests, we can conclude that the learned units do not correspond to phonetic categories in any meaningful sense of the term.”
Virtual Phone Discovery for Speech Synthesis Without Text (2019, background): “A brief description of the database and these metrics can be found in [30].”
The Zero Resource Speech Challenge 2021: Spoken Language Modelling (2021, unknown): “Previous work has shown that a low ABX score appears to be a very good predictor of good performance on TTS without T [30, 31, 32].”
Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling (2019, background): “The Zero Resource Speech Challenges (ZeroSpeech) 2015 [1], 2017 [2] and 2019 [3] precisely focus on this area.”