ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification

By Brecht Desplanques, Jenthe Thienpondt, Kris Demuynck. 2020

2,029 citations recorded by the source corpus.

Current speaker verification techniques rely on a neural network to extract speaker representations. The successful x-vector architecture is a Time Delay Neural Network (TDNN) that applies statistics pooling to project variable-length utterances into fixed-length speaker characterizing embeddings. In this paper, we propose multiple enhancements to this architecture based on recent trends in the related fields of face verification and computer vision. Firstly, the initial frame layers can be restructured into 1-dimensional Res2Net modules with impactful skip connections. Similarly to SE-ResNet, we introduce Squeeze-and-Excitation blocks in these modules to explicitly model channel interdependencies. The SE block expands the temporal context of the frame layer by rescaling the channels according to global properties of the recording. Secondly, neural networks are known to learn hierarchical features, with each layer operating on a different level of complexity. To leverage this complementary information, we aggregate and propagate features of different hierarchical levels. Finally, we improve the statistics pooling module with channel-dependent frame attention. This enables the network to focus on different subsets of frames during each of the channel's statistics estimation. The proposed ECAPA-TDNN architecture significantly outperforms state-of-the-art TDNN based systems on the VoxCeleb test sets and the 2019 VoxCeleb Speaker Recognition Challenge.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations (2021, methodology): “For the speaker embedding network, we borrow the neural architecture from a state-of-the-art speaker recognition network [14], which is based on 1D-convolutional neural networks (1D-CNN) with an attentive statistics pooling layer.”
The Idlab Voxsrc-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verification (2020, methodology): “We scale up the network compared to [12] by using 2048 feature channels and add a fourth SE-Res2Block with a dilation factor of 5 for optimized verification performance.”
NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis (2022, methodology): “The timbre tokens are extracted via cross-attention mechanism, where 50 trainable latent vectors are used a query, and features from multilayer feature aggregation (MFA) block of ECAPA-TDNN is transformed into key and value.”
SpeechBrain: A General-Purpose Speech Toolkit (2021, methodology): “[91, 92], such as x-vectors [32] and the recent ECAPA-TDNN embeddings [33].”
Wespeaker: A Research and Production Oriented Speaker Embedding Learning Toolkit (2022, background): “• ECAPA-TDNN, a modified version of TDNN, this is the champion system of VoxSRC 2020 [4].”
Powerset multi-class cross entropy loss for neural speaker diarization (2023, methodology): “All experiments reported in this paper rely on the pretrained ECAPA-TDNN model [19] from SpeechBrain [20] available at hf.co/speechbrain/spkrec-ecapa-voxceleb .”
Pyannote.audio 2.1 Speaker Diarization Pipeline: Principle, Benchmark, and Recipe (2023, methodology): “In particular, all experiments reported in Table 1 relies on the implementation of ECAPA-TDNN [19] available in SpeechBrain [20] because it was found to outperform three open-source alternatives.”