A Temporal Coherence Loss Function for Learning Unsupervised Acoustic Embeddings

By Gabriel Synnaeve, Emmanuel Dupoux. 2016

22 citations recorded by the source corpus.

Abstract We train neural networks of varying depth with a loss function which imposes the output representations to have a temporal profile which looks like that of phonemes. We show that a simple loss function which maximizes the dissimilarity between near frames and long distance frames helps to construct a speech embedding that improves phoneme discriminability, both within and across speakers, even though the loss function only uses within speaker information. However, with too deep an architecture, this loss function yields overfitting, suggesting the need for more data and/or regularization.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

wav2vec: Unsupervised Pre-training for Speech Recognition (2019, neutral): “…pre-training has focused on emotion recogniton (Lian et al., 2018), speaker identiﬁcation (Ravanelli & Bengio, 2018), phoneme discrimination (Synnaeve & Dupoux, 2016a; van den Oord et al., 2018) as well as transferring ASR representations from one language to another (Kunze et al., 2017).”
Performance-Efficiency Trade-Offs in Unsupervised Pre-Training for Speech Recognition (2021, background): “Besides ASR-focused works, there is significant interest in learning representations for other speech tasks (Synnaeve and Dupoux, 2016; Chung et al., 2018; Chuang et al., 2019; Song et al., 2019), music (Yang et al., 2021; Zhao and Guo, 2021), and general audio (Saeed et al., 2021; Gong et al.,…”
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation (2021, background): “In audio and speech processing, pre-training has focused on emotion recognition [24], speaker identification [32], phoneme discrimination [39, 27], transferring ASR representations from one language to another [17], unsupervised representations learning for speech [35], audio representation learning”
Learning Text Similarity with Siamese Recurrent Networks (2016, result): “This ratio was empirically determined but other studies have found a similar optimal ratio of negative to positive pairs in Siamese networks (Synnaeve and Dupoux, 2016).”