End-to-end text-dependent speaker verification

By Georg Heigold, Ignacio Moreno, Samy Bengio, Noam Shazeer. 2015

615 citations recorded by the source corpus.

In this paper we present a data-driven, integrated approach to speaker verification, which maps a test utterance and a few reference utterances directly to a single score for verification and jointly optimizes the system's components using the same evaluation protocol and metric as at test time. Such an approach will result in simple and efficient systems, requiring little domain-specific knowledge and making few model assumptions. We implement the idea by formulating the problem as a single neural network architecture, including the estimation of a speaker model on only a few utterances, and evaluate it on our internal "Ok Google" benchmark for text-dependent speaker verification. The proposed approach appears to be very effective for big data applications Like ours that require highly accurate, easy-to-maintain systems with a small footprint.

Subfields: Artificial Intelligence, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

Citations in the corpus

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (2018, methodology): “The network maps a sequence of log-mel spectrogram frames computed from a speech utterance of arbitrary length, to a fixed-dimensional embedding vector, known as d-vector [20, 9].”
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, methodology): “A pre-trained speaker verification model similar to the one proposed in [40] is used.”
Generalized End-to-End Loss for Speaker Verification (2017, methodology): “Model Embed Loss Multi Average Architecture Size Reader EER (%) (512, ) [13] 128 TE2E No 3.”
Deep Speaker: an End-to-End Neural Speaker Embedding System (2017, methodology): “Georg Heigold et al. [6] trained an LSTM for text-dependent speaker veriﬁcation, which acheived a 2% equal error rate (EER) on the “Ok Google” benchmark.”
Generating Multilingual Voices Using Speaker Space Translation Based on Bilingual Speaker Data (2020, background): “The Tacotron is conditioned on a speaker embedding d-vector [11] from the speaker encoder, trained separately following [12].”
Disentangling Correlated Speaker and Noise for Speech Synthesis via Data Augmentation and Adversarial Factorization (2019, methodology): “We next examine if zs can control the speaker identity of synthesized speech, using a text-independent 211 speaker verification system [27] to is compute speaker discriminative embeddings, called d-vectors [6], 212 from the reference and synthesized speech samples.”
Self-Attentive Speaker Embeddings for Text-Independent Speaker Verification (2018, background): “On the other hand, recent work demonstrates that more powerful SV systems can be built from directly training a speaker discriminative DNN [12, 13, 14, 15, 16, 17].”
Speaker Recognition for Multi-speaker Conversations Using X-vectors (2019, background): “, introduced an end-to-end system, trained on the phrase “OK Google,” that jointly learns an embedding along with a similarity metric to compare pairs of embeddings [8].”
Teacher-Student Training for Text-Independent Speaker Recognition (2018, background): “End-to-end training has been investigated in a number of studies, where SR systems were trained on distance-based objectives, such as pairwise cosine distance [10, 11], triplet loss [12, 13] and its generalised version [14].”
What Does the Speaker Embedding Encode? (2017, neutral): “Despite the proliferation of speaker embedding methods and their demonstrated effectiveness [6, 14, 16, 17, 18], a fundamental question remains largely unanswered: what properties are actually encoded in these embeddings?”
Speaker Recognition from Raw Waveform with SincNet (2018, background): “DNNs have also been proposed for direct discriminative speaker classification, as witnessed by the recent literature on this topic [12–15].”
Speaker Diarization with LSTM (2017, background): “In recent years, neural network based audio embeddings (d-vectors) have seen wide-spread use in speaker veriﬁcation applications [7, 8, 9, 10, 11], often signiﬁcantly outperforming previously state-of-the-art techniques based on i-vectors.”
Attentive Statistics Pooling for Deep Speaker Embedding (2018, methodology): “In fixed-phrase textdependent speaker verification, an end-to-end neural networkbased method has been proposed [6] in which Long Short-Term Memory (LSTM) with a single output from the last frame is used to obtain utterance-level speaker features, and it has outperformed conventional i-vector extract”
X-Vectors: Robust DNN Embeddings for Speaker Recognition (2018, background): “This has potential to produce powerful, compact systems [13], that only require speaker labels to train.”
Deep Speaker Embeddings for Short-Duration Speaker Verification (2017, methodology): “In [7] the authors simply averaged the hidden activations to obtain a single feature.”
End-to-End attention based text-dependent speaker verification (2016, methodology): “Another contribution of this work is the CNN extracted frame-level features are smartly combined through an attention mechanism to generate an utterance-level speaker representation, in stead of just equally weighting and averaging all the frames [5, 6].”