End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors

By Shota Horiguchi, Yusuke Fujita, Shinji Watanabe, Yawen Xue, Kenji Nagamatsu. 2020

230 citations recorded by the source corpus.

End-to-end speaker diarization for an unknown number of speakers is addressed in this paper. Recently proposed end-to-end speaker diarization outperformed conventional clustering-based speaker diarization, but it has one drawback: it is less flexible in terms of the number of speakers. This paper proposes a method for encoder-decoder based attractor calculation (EDA), which first generates a flexible number of attractors from a speech embedding sequence. Then, the generated multiple attractors are multiplied by the speech embedding sequence to produce the same number of speaker activities. The speech embedding sequence is extracted using the conventional self-attentive end-to-end neural speaker diarization (SA-EEND) network. In a two-speaker condition, our method achieved a 2.69 % diarization error rate (DER) on simulated mixtures and a 8.07 % DER on the two-speaker subset of CALLHOME, while vanilla SA-EEND attained 4.56 % and 9.54 %, respectively. In unknown numbers of speakers conditions, our method attained a 15.29 % DER on CALLHOME, while the x-vector-based clustering method achieved a 19.43 % DER.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Powerset multi-class cross entropy loss for neural speaker diarization (2023, background): “…is larger in test data than during training – additional mechanisms such as encoder-decoder attractors were proposed to partially solve this issue [6]; • training such approaches is extremely data hungry as each conversation in the training set constitutes a single training sample – one has to…”
Neural Speaker Diarization with Speaker-Wise Chain Rule (2020, background): “Encoder-decoder based attractor [37] is another recent approach to the variable number of speakers based on deep attractor networks [38].”
Encoder-Decoder Based Attractor Calculation for End-to-End Neural Diarization (2021, methodology): “One is an attractor-based approach [14], and the other is a speaker-wise conditional EEND (SC-EEND) [15], [16].”
A Review of Speaker Diarization: Recent Advances with Deep Learning (2021, methodology): “line version of EDA-EEND [169] by carrying the hidden state of the LSTM-encoder to generate attractors block by block.”
Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors (2021, methodology): “There are a few recent methods that can handle a ﬂexible number of speakers [10, 11, 12], but since they are trained in a fully supervised manner, they still face the problem of not being able to produce a larger number of speakers than that observed during training.”