Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech

By Keisuke Kinoshita, Marc Delcroix, Naohiro Tawara. 2021

87 citations recorded by the source corpus.

Recently, we proposed a novel speaker diarization method called End-to-End-Neural-Diarization-vector clustering (EEND-vector clustering) that integrates clustering-based and end-to-end neural network-based diarization approaches into one framework. The proposed method combines advantages of both frameworks, i.e. high diarization performance and handling of overlapped speech based on EEND, and robust handling of long recordings with an arbitrary number of speakers based on clustering-based approaches. However, the method was only evaluated so far on simulated 2-speaker meeting-like data. This paper is to (1) report recent advances we made to this framework, including newly introduced robust constrained clustering algorithms, and (2) experimentally show that the method can now significantly outperform competitive diarization methods such as Encoder-Decoder Attractor (EDA)-EEND, on CALLHOME data which comprises real conversational speech data including overlapped speech and an arbitrary number of speakers. By further analyzing the experimental results, this paper also discusses pros and cons of the proposed method and reveals potential for further improvement. A set of the code to reproduce the results is available at https://github.com/nttcslab-sp/EEND-vector-clustering.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Zhuo Chen
3. Shujie Liu
4. Shinji Watanabe
5. Yu Wu
6. Takuya Yoshioka
7. Sanyuan Chen
8. Dong Yu

In Signal Processing

1. Jinyu Li
2. Shinji Watanabe
3. Zhuo Chen
4. Takuya Yoshioka
5. Shujie Liu
6. Yu Wu
7. Sanyuan Chen
8. Dong Yu

Citations in the corpus

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, methodology): “Following the data usage in [64], we split the CALLHOME dataset into two parts.”
• Powerset multi-class cross entropy loss for neural speaker diarization (2023, background): “Therefore, a few recent works took a step back to borrow the best of both worlds (BoBW, i.e. multi-stage and end-to-end approaches) [8, 9].”
• Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors (2021, methodology): “One possible choice is to use COP-Kmeans clustering [25], which is used in EEND-vector clustering [26, 27], but this has a difficulty in the case where cannot-link constraints have to be satisfied; the algorithm sometimes results in no solution.”