Generalized End-to-End Loss for Speaker Verification

By Li Wan, Quan Wang, Alan Papir, Ignacio Lopez-Moreno. 2017

1,063 citations recorded by the source corpus.

In this paper, we propose a new loss function called generalized end-to-end (GE2E) loss, which makes the training of speaker verification models more efficient than our previous tuple-based end-to-end (TE2E) loss function. Unlike TE2E, the GE2E loss function updates the network in a way that emphasizes examples that are difficult to verify at each step of the training process. Additionally, the GE2E loss does not require an initial stage of example selection. With these properties, our model with the new loss function decreases speaker verification EER by more than 10%, while reducing the training time by 60% at the same time. We also introduce the MultiReader technique, which allows us to do domain adaptation - training a more accurate model that supports multiple keywords (i.e., “OK Google” and “Hey Google”) as well as multiple dialects.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Junichi Yamagishi
5. James R. Glass
6. Hung-yi Lee
7. Wei-Ning Hsu
8. Dong Yu

In Speech & Audio

1. Xu Tan
2. Hung-yi Lee
3. Wei-Ning Hsu
4. Tao Qin
5. Junichi Yamagishi
6. Sheng Zhao
7. Tie-Yan Liu
8. James R. Glass

In Signal Processing

1. Junichi Yamagishi
2. Xu Tan
3. Tao Qin
4. Hung-yi Lee
5. Tie-Yan Liu
6. Wei-Ning Hsu
7. James R. Glass
8. Shinji Watanabe

Citations in the corpus

• FedSpeech: Federated Text-to-Speech with Continual Learning (2021, methodology): “For each task, we derive the high-level representation vector that summarizes the characteristics of the speaker’s voice using the encoder 3 implemented from Wan et al. [ 2018 ] .”
• YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, positive): “Meanwhile, Tacotron 2 [7] was adapted using external speaker embeddings extracted from a trained speaker encoder using a generalized end-to-end loss (GE2E) [8], allowing for speech generation that resembles the target speaker [1].”
• Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance (2021, methodology): “(2018), we train a speaker encoder with GE2E loss (Wan et al., 2018) on the speaker verification dataset and use the speaker encoder to condition speaker-dependent modules.”
• Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech (2021, methodology): “In our experiments, we use GE2E model [23] as the speaker encoder, which is the same speaker verification model we used to extract d-vectors (Section VI-B).”
• AdaSpeech: Adaptive Text to Speech for Custom Voice (2021, background): “Some works only fine-tune the speaker embedding (Arik et al., 2018; Chen et al., 2018), or train a speaker encoder module (Arik et al., 2018; Jia et al., 2018; Cooper et al., 2020; Li et al., 2017; Wan et al., 2018) that does not need fine-tuning during adaptation.”
• Sample Efficient Adaptive Text-to-Speech (2018, methodology): “These results are robust across speech datasets recorded under different conditions and, moreover, we demonstrate that the generated samples are capable of confusing the state-of-the-art text-independent speaker verification system (Wan et al., 2018).”
• Wavenet Based Low Rate Speech Coding (2017, methodology): “An objective speaker identification test was performed using a neural network based speaker identification model [31].”
• Icassp 2022 Deep Noise Suppression Challenge (2022, background): “Participants could leverage other state-of-the-art speaker models [18, 19, 16] for extracting speaker embeddings [18, 19].”
• SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model (2021, methodology): “Following [4], we compute SECS using the speaker encoder of the Resemblyzer [7, 34] package; thus, allowing comparison with those studies.”
• The Multi-Speaker Multi-Style Voice Cloning Challenge 2021 (2021, positive): “D-vector [34], ResNet34 [35] and x-vector [36] were used in T07, T16 and T23 respectively.”
• Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speech (2021, background): “[15] is a speaker representation generated by a neural network pretrained on a speaker classification task, and is widely used in speaker verification systems.”
• Any-to-Many Voice Conversion With Location-Relative Sequence-to-Sequence Modeling (2020, background): “As in [35], [36], we train the speaker encoder to optimize a generalized end-to-end (GE2E) speaker verification loss.”
• SpeakerBeam: Speaker Aware Neural Network for Target Speaker Extraction in Speech Mixtures (2019, methodology): “The method introduced in [33] proposes concatenating a dvector [34] extracted from the adaptation utterance with one of”
• Zero-Shot Voice Style Transfer with Only Autoencoder Loss (2019, methodology): “The speaker encoder is pre-trained on the GE2E loss (Wan et al., 2018) (the softmax loss version), which maximizes the embedding similarity among different utterances of the same speaker, and minimizes the similarity among different speakers.”
• Translatotron 2: Robust direct speech-to-speech translation (2021, methodology): “We explored into alternative objective evaluation using separately trained speaker encoders, such as d-vector (Wan et al., 2018).”