Exploring wav2vec 2.0 on speaker verification and language identification

By Zhiyun Fan, Meng Li, Shiyu Zhou, Bo Xu. 2020

244 citations recorded by the source corpus.

Wav2vec 2.0 is a recently proposed self-supervised framework for speech representation learning. It follows a two-stage training process of pre-training and fine-tuning, and performs well in speech recognition tasks especially ultra-low resource cases. In this work, we attempt to extend self-supervised framework to speaker verification and language identification. First, we use some preliminary experiments to indicate that wav2vec 2.0 can capture the information about the speaker and language. Then we demonstrate the effectiveness of wav2vec 2.0 on the two tasks respectively. For speaker verification, we obtain a new state-of-the-art result, Equal Error Rate (EER) of 3.61% on the VoxCeleb1 dataset. For language identification, we obtain an EER of 12.02% on 1 second condition and an EER of 3.47% on full-length condition of the AP17-OLR dataset. Finally, we utilize one model to achieve the unified modeling by the multi-task learning for the two tasks.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised Speech Recognition (2021, background): “There has been a lot of recent work in this direction which has shown strong performance in extremely low-labeled data setups across a range of languages (Conneau et al., 2020) and tasks (Fan et al., 2021; Pepino et al., 2021; Wang et al., 2021).”
Direct Speech-to-Speech Translation With Discrete Units (2021, background): “The recent success in self-supervised learning for speech has demonstrated that speech representations learned from a large unlabelled speech corpus can lead to impressive performance on a variety of downstream tasks [5] including ASR [6, 7], speaker and language identification [8], emotion recognit”
SUPERB: Speech processing Universal PERformance Benchmark (2021, background): “…have investigated these SSL models’ capabilities on tasks including phoneme classiﬁcation [11, 7], speaker identiﬁcation [7, 8], speaker ver-iﬁcation [17], emotion recognition [15], ASR [9, 12, 10, 16], speech translation [7], spoken language understanding [18], voice conversion [19] and TTS [20].”
Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations (2021, neutral): “Note that this is in accordance with the previous observation in [18].”
Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation (2022, unknown): “0, have been applied to automatic speech recognition [12, 13], mispronunciation detection [36, 37], speaker recognition [38, 39] and emotion recognition [40].”
Scaling Speech Technology to 1, 000+ Languages (2023, methodology): “Li et al. [2021] built multilingual ASR for up to 15 languages, Pratap et al. [2020a], Lugosch et al. [2022], Tjandra et al. [2022b] trained and explored several strategies for 50 + languages.”
SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities (2022, unknown): “…(Liu et al., 2020b; Schneider et al., 2019; Ling and Liu, 2020; Ravanelli et al., 2020; Hsu et al., 2021; Chang et al., 2021), speaker veriﬁcation (Fan et al., 2020), speaker identiﬁcation (Chung et al., 2019; Liu et al., 2020c), emotion recognition (Macary et al., 2021), speech translation…”
Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling (2022, background): “…or very competitive results on automatic speech recognition (Baevski et al. 2020), phoneme classiﬁcation (Chung et al. 2019), speaker identiﬁcation (Fan et al. 2021), emotion recognition (Pascual et al. 2019), speech translation (Chung et al. 2019), spoken language understanding (Lai et al.…”