The Zero Resource Speech Challenge 2021: Spoken Language Modelling

By Ewan Dunbar, Mathieu Bernard, Nicolas Hamilakis, Tu Nguyen, Maureen de Seyssel, Patricia Roz'e, Morgane Rivière, Evgeny Kharitonov, Emmanuel Dupoux. 2021

68 citations recorded by the source corpus.

We present the Zero Resource Speech Challenge 2021, which asks participants to learn a language model directly from audio, without any text or labels. The challenge is based on the Libri-light dataset, which provides up to 60k hours of audio from English audio books without any associated text. We provide a pipeline baseline system consisting on an encoder based on contrastive predictive coding (CPC), a quantizer ($k$-means) and a standard language model (BERT or LSTM). The metrics evaluate the learned representations at the acoustic (ABX discrimination), lexical (spot-the-word), syntactic (acceptability judgment) and semantic levels (similarity judgment). We present an overview of the eight submitted systems from four groups and discuss the main results.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

AudioLM: A Language Modeling Approach to Audio Generation (2022, methodology): “We use two zero-shot metrics, sWUGGY and sBLIMP, introduced in the ZeroResource Challenge 2021 [13].”
Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision (2023, background): “At its core, SPEAR-TTS leverages recent advances in the “textless” modeling of spoken language (Lakhotia et al., 2021; Dunbar et al., 2021; Polyak et al., 2021; Kreuk et al., 2021; Kharitonov et al., 2022; Nguyen et al., 2022; Borsos et al., 2022).”
Are Discrete Units Necessary for Spoken Language Modeling? (2022, background): “The trained models displayed better-than-chance performances on nearly all the evaluation metrics of the challenge [9], [10].”
A Brief Overview of Unsupervised Neural Speech Representation Learning (2022, background): “The longstanding zero resource speech challenge (ZeroSpeech) offers a new set of tasks for each edition (Versteegh et al. 2015; Dunbar et al. 2017, 2019, 2020, 2021) usually featuring a minimal-pair ABX task (Schatz et al. 2013, 2014).”