Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation

By Felix Kreuk, Joseph Keshet, Yossi Adi. 2020

91 citations recorded by the source corpus.

We propose a self-supervised representation learning model for the task of unsupervised phoneme boundary detection. The model is a convolutional neural network that operates directly on the raw waveform. It is optimized to identify spectral changes in the signal using the Noise-Contrastive Estimation principle. At test time, a peak detection algorithm is applied over the model outputs to produce the final boundaries. As such, the proposed model is trained in a fully unsupervised manner with no manual annotations in the form of target boundaries nor phonetic transcriptions. We compare the proposed approach to several unsupervised baselines using both TIMIT and Buckeye corpora. Results suggest that our approach surpasses the baseline models and reaches state-of-the-art performance on both data sets. Furthermore, we experimented with expanding the training set with additional examples from the Librispeech corpus. We evaluated the resulting model on distributions and languages that were not seen during the training phase (English, Hebrew and German) and showed that utilizing additional untranscribed data is beneficial for model performance.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, methodology): “SSL methods have shown remarkable results for ASR [2, 3], phoneme segmentation [13], and GSLM [5].”
AudioGen: Textually Guided Audio Generation (2022, background): “…shown remarkable results for automatic speech recognition (Schneider et al., 2019; Baevski et al., 2020; Wang et al., 2021), phoneme segmentation (Kreuk et al., 2020), and audio compression (Zeghidour et al., 2021; Polyak et al., 2021). van den Oord et al. (2018) and Schneider et al. (2019)…”
A Brief Overview of Unsupervised Neural Speech Representation Learning (2022, background): “In practice, this is often achieved implicitly, by learning segment boundaries or by taking repeated quantized values to belong to the same segment (Kamper and van Niekerk 2021; Chorowski et al. 2019a; Michel et al. 2017; Kreuk, Keshet, and Adi 2020; Wang, Chung, and Lee 2017; Dieleman et al. 2021).”