In defence of metric learning for speaker recognition

By Joon Son Chung, Jaesung Huh, Seongkyu Mun, Minjae Lee, Hee-Soo Heo, Soyeon Choe, Chiheon Ham, Sung-Ye Jung, Bong-Jin Lee, Icksang Han. 2020

499 citations recorded by the source corpus.

The objective of this paper is 'open-set' speaker recognition of unseen speakers, where ideal embeddings should be able to condense information into a compact utterance-level representation that has small intra-speaker and large inter-speaker distance. A popular belief in speaker recognition is that networks trained with classification objectives outperform metric learning methods. In this paper, we present an extensive evaluation of most popular loss functions for speaker recognition on the VoxCeleb dataset. We demonstrate that the vanilla triplet loss shows competitive performance compared to classification-based losses, and those trained with our proposed metric learning objective outperform state-of-the-art methods.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[38] Joon Son Chung, Jaesung Huh, Seongkyu Mun, Minjae Lee, Hee Soo Heo, Soyeon Choe, Chiheon Ham, Sunghwan Jung, Bong-Jin Lee, and Icksang Han.”
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, methodology): “As speaker encoder, we use the H/ASP model [29] publicly available, that was trained with the Prototypical Angular [30] plus Softmax loss functions in the VoxCeleb 2 [31] dataset.”
SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model (2021, methodology): “Optimization was carried out using the Angular Prototypical [13] loss function different than the original work.”
HierSpeech: Bridging the Gap between Text and Speech by Hierarchical Variational Inference using Self-supervised Representations for Speech Synthesis (2022, unknown): “We used a pre-trained ASV model (Chung et al., 2020).”
Speaker recognition based on deep learning: An overview (2020, unknown): “Similar works were also conducted in [160, 161, 162].”
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (2021, background): “E n is borrowed from [72] and E ac is a ResNetSE34 borrowed from [13].”
Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020 (2020, methodology): “The network architecture is identical to that used in [7] except for the input dimension, and we refer to this configuration as Q / SAP in the results.”
Zero-Shot Text-to-Speech Synthesis Conditioned Using Self-Supervised Speech Representation Model (2023, methodology): “The 256-dimensional x-vector was extracted from 16 kHz speech signals by using the speaker identification model based on FastResNet-34 with self-attentive pooling (SAP) trained with angular prototypical loss [28].”
Spot the conversation: speaker diarisation in the wild (2020, methodology): “A pre-trained speaker recognition model [33] is used to verify the identity of speech that comes from off-screen speakers.”
HierSpeech++: Bridging the Gap Between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-Shot Speech Synthesis (2023, unknown): “In [85], the e ﬀ ectiveness of metric learning in automatic speaker veriﬁcation was demonstrated.”