VoxCeleb2: Deep Speaker Recognition
By Joon Son Chung, Arsha Nagrani, Andrew Zisserman. 2018
2,807 citations recorded by the source corpus.
The objective of this paper is speaker recognition under noisy and unconstrained conditions. We make two key contributions. First, we introduce a very large-scale audio-visual speaker recognition dataset collected from open-source media. Using a fully automated pipeline, we curate VoxCeleb2 which contains over a million utterances from over 6,000 speakers. This is several times larger than any publicly available speaker recognition dataset. Second, we develop and compare Convolutional Neural Network (CNN) models and training strategies that can effectively recognise identities from voice under various conditions. The models trained on the VoxCeleb2 dataset surpass the performance of previous works on a benchmark dataset by a significant margin.
Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Evaluation & Benchmarks
- Andrew Zisserman
- Hung-yi Lee
- Andrea Vedaldi
- Lior Wolf
- Yu Zhang
- Wei-Ning Hsu
- Jun-Yan Zhu
- Yoshua Bengio
In Speech & Audio
In Signal Processing
Citations in the corpus
- VOXLINGUA107: A Dataset for Spoken Language Recognition (2020, background): “The VoxCeleb 1&2 [8, 9] corpora are popular free datasets for training speaker recognition models, containing automatically retrieved data from YouTube.”
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (2018, methodology): “We made use of three additional training sets: (1) LibriSpeech Other, which contains 461 hours of speech from a set of 1,166 speakers disjoint from those in the clean subsets, (2) VoxCeleb [11], and (3) VoxCeleb2 [6] which contain 139K utterances from 1,211 speakers, and 1.”
- YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, methodology): “As speaker encoder, we use the H/ASP model [29] publicly available, that was trained with the Prototypical Angular [30] plus Softmax loss functions in the VoxCeleb 2 [31] dataset.”
- Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance (2021, methodology): “To extract the speaker embedding eS from each utterance, we train a speaker encoder on VoxCeleb2 (Chung et al., 2018), which is a speaker verification dataset that contains more than 1M utterances of 6112 speakers.”
- Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech (2021, methodology): “The model was trained on a combination of subsets of the three datasets: Librispeech train-other-500 [24], VoxCeleb1 Dev A-D [25], and VoxCeleb2 Dev A-H [26], which is 3201 hours of utterances in total from 8371 speakers3.”
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, neutral): “2) Datasets: VoxCeleb1 [20] and VoxCeleb2 [54] datasets are used in our experiments for speaker verification.”
- Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, methodology): “It was trained on the VoxCeleb2 [47] dataset, achieving a 7.”
- Icassp 2022 Deep Noise Suppression Challenge (2022, methodology): “The baseline speaker embedding extractor is RawNet2 [16] trained on wideband audio from VoxCeleb2 [17].”
- Textless Speech-to-Speech Translation on Real Data (2021, positive): “The 256dimensional speaker embedding, which remains fixed during the S2UT model training, is extracted from a speaker verification model pre-trained on VoxCeleb2 (Chung et al., 2018).”
- Audio-Visual Scene Analysis with Self-Supervised Multisensory Features (2018, methodology): “For this, we combined the VoxCeleb and VoxCeleb2 [77] datasets (approx.”
- Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings (2019, methodology): “Data : Following [13], we trained speaker verification systems on VoxCeleb1+2 [26, 27].”
- MelNet: A Generative Model for Audio in the Frequency Domain (2019, methodology): “To demonstrate MelNet’s capacity to model distributions with significantly more variation, we utilize the VoxCeleb2 dataset [8].”
- WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “There are many public speech datasets with speaker metadata that could potentially be used to construct a benchmark, e.g., LibriSpeech (Panayotov et al., 2015), the Speech Accent Archive (Weinberger, 2015), VoxCeleb2 (Chung et al., 2018), the Spoken Wikipedia Corpus (Baumann et al., 2019), and Commo”
- Pyannote.Audio: Neural Building Blocks for Speaker Diarization (2019, methodology): “While speaker embeddings were trained and tested on VoxCeleb [19], all other models (including the full diarization pipeline) were trained, tuned, and tested on three different datasets, covering a wide range of domains: meetings for AMI [20], broadcast news for ETAPE [21], and up to 11 different…”
- SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model (2021, methodology): “Our speaker encoder was trained with all partitions of the LibriSpeech dataset [25], the English version of Common Voice [26], the VCTK and VoxCeleb (v1 and v2) datasets [27], totaling approximately 25k speakers.”
- SpeechBrain: A General-Purpose Speech Toolkit (2021, methodology): “Table 4 reports the performance achieved on a speaker verification task with models trained on VoxCeleb2 [35] and tested on VoxCeleb1-clean [34].”
- Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition (2023, methodology): “…following datasets to train Speaker Embedding extractor (TitaNet) and ASR models (En Fast Conformer, En-Es Fast Con-former): • TitaNet: Voxceleb1&2 [22] and NIST-SRE 2004-2008 (LDC2009E100) • En Fast Conformer : 960 hrs of LibriSpeech(LS) [23] • En-Es Fast Conformer: LS and 776 hrs of MLS [24]…”
- ICASSP 2021 Deep Noise Suppression Challenge (2020, methodology): “We used VoxCeleb2 [26] corpus for neighboring speakers.”
- PIE (2020, methodology): “2017], trained on the VoxCeleb dataset [Chung et al. 2018].”