End-to-End attention based text-dependent speaker verification

By Shi-Xiong Zhang, Zhuo Chen, Yong Zhao, Jinyu Li, Yifan Gong. 2016

181 citations recorded by the source corpus.

A new type of End-to-End system for text-dependent speaker verification is presented in this paper. Previously, using the phonetic discriminate/speaker discriminate DNN as a feature extractor for speaker verification has shown promising results. The extracted frame-level (bottleneck, posterior or d-vector) features are equally weighted and aggregated to compute an utterance-level speaker representation (d-vector or i-vector). In this work we use a speaker discriminate CNN to extract the noise-robust frame-level features. These features are smartly combined to form an utterance-level speaker vector through an attention mechanism. The proposed attention model takes the speaker discriminate information and the phonetic information to learn the weights. The whole system, including the CNN and attention model, is joint optimized using an end-to-end criterion. The training algorithm imitates exactly the evaluation process — directly mapping a test utterance and a few target speaker utterances into a single verification score. The algorithm can smartly select the most similar impostor for each target speaker to train the network. We demonstrated the effectiveness of the proposed end-to-end system on Windows 10 “Hey Cortana” speaker verification task.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Generalized End-to-End Loss for Speaker Verification (2017, background): “In recent years, more efforts have been focusing on using neural networks for speaker verification, while the most successful systems use end-to-end training [8, 9, 10, 11, 12].”
Deep Speaker Embeddings for Short-Duration Speaker Verification (2017, background): “Recently there have been several efforts to use deep neural network to learn speaker embeddings [6, 7, 8].”
Speaker recognition based on deep learning: An overview (2020, methodology): “Some similar idea can also be found in [149].”
Speech Processing for Digital Home Assistants: Combining signal processing with deep-learning techniques (2019, background): “Speaker identification algorithms can be text-dependent, typically based on the wake-up keyword, [58], [59] or text independent [60], [61], and run locally on the device or on the server.”
Deep Speaker Feature Learning for Text-Independent Speaker Verification (2017, methodology): “[16] D. Povey, X. Zhang, and S. Khudanpur, “Parallel training of dnns with natural gradient and parameter averaging,” arXiv preprint arXiv:1410.7455, 2014.”
Teacher-Student Training for Text-Independent Speaker Recognition (2018, background): “End-to-end training has been investigated in a number of studies, where SR systems were trained on distance-based objectives, such as pairwise cosine distance [10, 11], triplet loss [12, 13] and its generalised version [14].”
Self-Attentive Speaker Embeddings for Text-Independent Speaker Verification (2018, background): “Zhang et al, proposed using an attention model to combine the frame-level features for a text-dependent SV application [14].”
X-Vectors: Robust DNN Embeddings for Speaker Recognition (2018, background): “As a result, these systems have found success leveraging large proprietary datasets [13, 17, 18].”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “machine translation [19], [170], [249], attention was later successfully applied to various tasks, like image captioning [253], [277], speech recognition [36] and speaker verification [285].”