VoxCeleb: A Large-Scale Speaker Identification Dataset
By Arsha Nagrani, Joon Son Chung, Andrew Zisserman. 2017
2,732 citations recorded by the source corpus.
Most existing datasets for speaker identification contain samples obtained under quite constrained conditions, and are usually hand-annotated, hence limited in size. The goal of this paper is to generate a large scale text-independent speaker identification dataset collected 'in the wild'. We make two contributions. First, we propose a fully automated pipeline based on computer vision techniques to create the dataset from open-source media. Our pipeline involves obtaining videos from YouTube; performing active speaker verification using a two-stream synchronization Convolutional Neural Network (CNN), and confirming the identity of the speaker using CNN based facial recognition. We use this pipeline to curate VoxCeleb which contains hundreds of thousands of 'real world' utterances for over 1,000 celebrities. Our second contribution is to apply and compare various state of the art speaker identification techniques on our dataset to establish baseline performance. We show that a CNN based architecture obtains the best performance for both identification and verification.
Subfields: Artificial Intelligence, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Speech & Audio
In Signal Processing
Citations in the corpus
- BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, background): “The benchmark includes a variety of tasks such as speech emotion recognition [85], [86], speaker identification [87], and language identification [88], but specifically excludes tasks that are focused on the meaning of words.”
- VOXLINGUA107: A Dataset for Spoken Language Recognition (2020, background): “The VoxCeleb 1&2 [8, 9] corpora are popular free datasets for training speaker recognition models, containing automatically retrieved data from YouTube.”
- MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, methodology): “[86] Arsha Nagrani, Joon Son Chung, and Andrew Zisserman.”
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (2018, methodology): “We made use of three additional training sets: (1) LibriSpeech Other, which contains 461 hours of speech from a set of 1,166 speakers disjoint from those in the clean subsets, (2) VoxCeleb [11], and (3) VoxCeleb2 [6] which contain 139K utterances from 1,211 speakers, and 1.”
- YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, neutral): “This model was chosen for achieving state-of-the-art results in VoxCeleb 1 (Nagrani et al., 2017) test subset.”
- Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech (2021, methodology): “The model was trained on a combination of subsets of the three datasets: Librispeech train-other-500 [24], VoxCeleb1 Dev A-D [25], and VoxCeleb2 Dev A-H [26], which is 3201 hours of utterances in total from 8371 speakers3.”
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, background): “Compared with the Fbank feature, the representations from WavLM Large achieve over 35% relative EER improvement on all three trials for the VoxCeleb1 evaluation set.”
- SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing (2021, methodology): “Dataset and Evaluation Metrics We use the of-ficial split of the VoxCeleb1 dataset (Nagrani et al., 2017) for the SID task, where the test set contains 8,251 utterances from these 1,251 celebrities.”
- Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, background): “4% Equal Error Rate (EER) for speaker verification on the test split of the VoxCeleb1 [48] dataset.”
- Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System (2018, neutral): “It can be used for both speaker identification and verification [35].”
- VoxCeleb2: Deep Speaker Recognition (2018, methodology): “, from telephone calls [23, 24] or acoustic laboratories [25, 26, 27]), (ii) they are manually annotated and hence limited in size [8], or (iii) not freely available to the speaker community [28, 27] (see [7] for a full review of existing datasets).”
- Fitting New Speakers Based on a Short Untranscribed Sample (2018, methodology): “Due to the expected training time, we focus in this submission on a subset we call “Libri-15GB”, which is The VoxCeleb dataset (Nagrani et al., 2017) is a compilation of YouTube urls and time stamps, which were obtained using an automatic pipeline, which consists of video-based active speaker…”
- Audio-Visual Scene Analysis with Self-Supervised Multisensory Features (2018, methodology): “Training We evaluated our model on the task of separating speech sounds using the VoxCeleb dataset [71].”
- Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings (2019, methodology): “Data : Following [13], we trained speaker verification systems on VoxCeleb1+2 [26, 27].”
- Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms (2020, methodology): “We used the VoxCeleb2 dataset [25] for training, and we used the VoxCeleb1 dataset [29] to evaluate various protocols.”
- An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “VoxCeleb [185] 2017 1,251 Continuous sentences Not specifiedb Not specifiedb Videos in the wild from [195] (690 males) (153,516 utterances, Youtube 352 hours) Challenging multi-speaker acoustic environments”
- Interpretable Convolutional Filters with SincNet (2018, background): “Most prior attempts exploit magnitude spectrogram features [22, 23, 24, 46, 47, 48].”