Self-Attentive Speaker Embeddings for Text-Independent Speaker Verification

By Yingke Zhu, Tom Ko, David Snyder, Brian Kan-Wing Mak, Daniel Povey. 2018

269 citations recorded by the source corpus.

This paper introduces a new method to extract speaker embed-dings from a deep neural network (DNN) for text-independent speaker verification. Usually, speaker embeddings are extracted from a speaker-classification DNN that averages the hidden vectors over the frames of a speaker; the hidden vectors produced from all the frames are assumed to be equally important. We relax this assumption and compute the speaker embedding as a weighted average of a speaker’s frame-level hidden vectors, and their weights are automatically determined by a self-attention mechanism. The effect of multiple attention heads are also investigated to capture different aspects of a speaker’s input speech. Finally, a PLDA classifier is used to compare pairs of embeddings. The proposed self-attentive speaker embedding system is compared with a strong DNN embedding baseline on NIST SRE 2016. We find that the self-attentive embeddings achieve superior performance. Moreover, the improvement produced by the self-attentive speaker embeddings is consistent with both short and long testing utterances.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hung-yi Lee
2. Sakriani Watiasri Sakti
3. Satoshi Nakamura
4. Andros Tjandra
5. Da-Rong Liu
6. Yi-Chen Chen
7. Sung-Feng Huang
8. Kris Demuynck

In Representation & Self-Supervised Learning

1. Sakriani Watiasri Sakti
2. Satoshi Nakamura
3. Andros Tjandra
4. Yi-Chen Chen
5. Sung-Feng Huang
6. Da-Rong Liu
7. Seung-bin Kim
8. Po-Han Chi

In Speech & Audio

1. Hung-yi Lee
2. Sakriani Watiasri Sakti
3. Andros Tjandra
4. Satoshi Nakamura
5. Sung-Feng Huang
6. Yi-Chen Chen
7. Da-Rong Liu
8. Thomas Hain

In Signal Processing

1. Hung-yi Lee
2. Sakriani Watiasri Sakti
3. Satoshi Nakamura
4. Andros Tjandra
5. Yi-Chen Chen
6. Sung-Feng Huang
7. Da-Rong Liu
8. Kris Demuynck

Citations in the corpus

• VOXLINGUA107: A Dataset for Spoken Language Recognition (2020, methodology): “For temporal pooling, we use multi-head attention, similarly to the method described in [19].”
• The Idlab Voxsrc-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verification (2020, background): “The statistics pooling layer that maps the variable length input to a fixed-length representation can be enhanced through a temporal attention mechanism [6, 7].”
• The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021 (2021, methodology): “Furthermore, a multihead mechanism was introduced to increase the diversity of attention, such as multi-head self-attentive (MHSA) pooling [17] and self multi-head attention (MHA) pooling [18].”
• Machine Speech Chain (2020, background): “Since the deep learning approach become more popular, several deep learning architectures (DeepSpeaker [49], [50]) have been”
• Unsupervised Adaptation of Acoustic Models for ASR Using Utterance-Level Embeddings from Squeeze and Excitation Networks (2019, methodology): “Recently, attention-based pooling is used to generate speaker embeddings for speaker recognition and verification tasks [35], [38].”
• Improved RawNet with Filter-wise Rescaling for Text-independent Speaker Verification using Raw Waveforms (2020, methodology): “However, applying an attention mechanism to speaker verification is more focused on attentive pooling than applying such mechanisms to intermediate feature maps in image domain tasks [7, 16, 17].”
• SpeechNet: A Universal Modularized Model for Speech Processing Tasks (2021, methodology): “We apply self-attention pooling (Lin et al., 2017; Zhu et al., 2018) on Speaker Encoder to obtain the speaker embedding vector.”