Attention-Based Models for Text-Dependent Speaker Verification

By F. R. Chowdhury, Quan Wang, Ignacio Lopez-Moreno, Li Wan. 2017

175 citations recorded by the source corpus.

Attention-based models have recently shown great performance on a range of tasks, such as speech recognition, machine translation, and image captioning due to their ability to summarize relevant information that expands through the entire length of an input sequence. In this paper, we analyze the usage of attention mechanisms to the problem of sequence summarization in our end-to-end text-dependent speaker recognition system. We explore different topologies and their variants of the attention layer, and compare different pooling methods on the attention weights. Ultimately, we show that attention-based models can improves the Equal Error Rate (EER) of our speaker verification system by relatively 14% compared to our non-attention LSTM baseline model.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System (2018, methodology): “We implement the SAP layer similar to [27, 28, 29].”
Speaker Diarization with LSTM (2017, background): “In recent years, neural network based audio embeddings (dvectors) have seen wide-spread use in speaker verification applications [7, 8, 9, 10, 11], often significantly outperforming previously state-of-the-art techniques based on i-vectors.”
In defence of metric learning for speaker recognition (2020, background): “Contrastive loss [25] and triplet loss [26] have been demonstrated promising performance on speaker recognition [27, 28] by optimising the distance metrics directly, but these methods require careful pair or triplet selection which can be time consuming and performance sensitive.”
Deep Speaker Embedding Extraction with Channel-Wise Feature Responses and Additive Supervision Softmax Loss Function (2019, methodology): “[6] introduced the different attention mechanisms into the LSTM model to explore different topologies and variants of the attention layer.”
Voxceleb: Large-scale speaker verification in the wild (2020, methodology): “A number of papers (Wan et al., 2018; Chowdhury et al., 2017) have also used LSTM-based front-end architectures, including the work by Heigold et al.”