Utterance-level Aggregation for Speaker Recognition in the Wild

By Weidi Xie, Arsha Nagrani, Joon Son Chung, Andrew Zisserman. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2019

370 citations recorded by the source corpus.

The objective of this paper is speaker recognition ‘in the wild’ – where utterances may be of variable length and also contain irrelevant signals. Crucial elements in the design of deep networks for this task are the type of trunk (frame level) network, and the method of temporal aggregation. We propose a powerful speaker recognition deep network, using a ‘thin-ResNet’ trunk architecture, and a dictionary-based NetVLAD or GhostVLAD layer to aggregate features across time, that can be trained end-to-end. We show that our network achieves state of the art performance by a significant margin on the VoxCeleb1 test set for speaker recognition, whilst requiring fewer parameters than previous methods. We also investigate the effect of utterance length on performance, and conclude that for ‘in the wild’ data, a longer length is beneficial.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f500a4da21f47725b0a180c41b56693a5db66252

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Vggsound: A Large-Scale Audio-Visual Dataset (2020, methodology): “To this end, we investigate different architectures, VGGish [3, 10] and ResNet [4] networks, as well as different aggregation approaches, global average pooling and NetVLAD [14, 15], for training deep CNNs on spectrograms extracted directly from the audio ﬁles with little pre-processing.”
Deep Speaker Embedding Extraction with Channel-Wise Feature Responses and Additive Supervision Softmax Loss Function (2019, background): “95 Xie et al.[23] Thin ResNet-34 Softmax VoxCeleb2 3.”
Voxceleb: Large-scale speaker verification in the wild (2020, methodology): “While only training on Voxceleb2, our Thin-ResNet-34 (Xie et al., 2019) achieves comparable performance to the model based on x-vectors trained on Voxceleb1&2 (EER 3.2 vs. EER 3.1).”
AutoSpeech: Neural Architecture Search for Speaker Recognition (2020, positive): “End-to-end speaker recognition systems [1, 2, 3, 4, 5, 6] have emerged in recent years and achieved the state-of-the-art performance.”
Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings (2019, methodology): “Several later studies [16, 19, 21, 22, 25, 27] suggest replacing the TDNN with variants of ResNet34, composed of 2D convolutions, as the encoder network.”
Scaling Effect of Self-Supervised Speech Models (2021, methodology): “The reason to use self-supervised models is twofold: i) it has a self-supervised pre-train loss that resembles to the cross-entropy loss in [7], which allows us to verify the power-law scaling property, ii) unlike supervised speech models trained on one particular classification task [8] [9], the se”
Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms (2020, background): “[27] Spectrogram Thin ResNet-34 GhostVLAD Softmax 512 5.”
Noise Robust TTS for Low Resource Speakers using Pre-trained Model and Speech Enhancement (2020, methodology): “We apply the approach in [14] to implement the speaker recognition model, which consists of a modified ResNet [15] in a fully convolutional way to extract framelevel features and a following GhostVLAD [16] layer for feature aggregation along the temporal axis.”
Exploring wav2vec 2.0 on speaker verification and language identification (2020, background): “Recently, neural networks trained with a large amount of labeled data can meet most industrial needs in the field of speech processing [1, 2, 3, 4, 5, 6].”
CN-Celeb: A Challenging Chinese Speaker Recognition Dataset (2019, methodology): “We used an off-the-shelf speaker recognition system [21] to perform this double check.”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “[10] extracted a compact speaker representation from an enrolment speech signal with the deep-learning-based method in [276] and obtained good performance for mixtures of two and”
Speaker recognition based on deep learning: An overview (2020, methodology): “For example, to reduce the number of parameters, [60] modified the standard ResNet-34 to a thin ResNet by cutting down the number of channels in each residual block.”
Delving into VoxCeleb: environment invariant speaker recognition (2019, background): “Residual networks [15] are used widely in image recognition and has recently been applied to speaker recognition [16, 17, 18].”