Diarization is Hard: Some Experiences and Lessons Learned for the JHU Team in the Inaugural DIHARD Challenge

By Gregory Sell, David Snyder, Alan McCree, Daniel Garcia-Romero, Jesus Villalba, Matthew Maciejewski, Vimal Manohar, Najim Dehak, Daniel Povey, Shinji Watanabe and 1 more. 2018

234 citations recorded by the source corpus.

We describe in this paper the experiences of the Johns Hopkins University team during the inaugural DIHARD diarization evaluation. This new task provided microphone recordings in a variety of difﬁcult conditions and challenged researchers to fully consider all speaker activity, without the currently typi-cal practices of unscored collars or ignored overlapping speaker segments. This paper explores several key aspects of currently state-of-the-art diarization methods, such as training data se-lection, signal bandwidth for feature extraction, representations of speech segments (i-vector versus x-vector), and domain-adaptive processing. In the end, our best system clustered x-vector embeddings trained on wideband microphone data followed by Variational-Bayesian reﬁnement, and a speech activity detector speciﬁcally trained for this task with in-domain data was found to be the best performing. After presenting these decisions and their ﬁnal result, we discuss lessons learned and remaining challenges within the lens of this new approach to diarization performance measurement.

Subfields: Artificial Intelligence, Biomedical Engineering, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

Junichi Yamagishi

In Speech & Audio

Citations in the corpus

Encoder-Decoder Based Attractor Calculation for End-to-End Neural Diarization (2021, methodology): “For speaker embeddings, i-vectors [18], [19], x-vectors [20]– [22], and d-vectors [6], [23]) have been explored.”
Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals (2020, methodology): “, speaker diarization [40, 1, 32]), which lose an end-to-end transduction function, unlike our proposed method.”
End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors (2020, background): “59 %, which outperformed the baseline [39] and the best pre-is2019-deadline system by the DI-IT team [40], but it could not beat the best post-is2019-deadline system by the BUT team [41].”
Learning Disentangled Phone and Speaker Representations in a Semi-Supervised VQ-VAE Paradigm (2020, methodology): “It used PLDA scoring and agglomerative hierarchical clustering [17].”
The Second DIHARD Diarization Challenge: Dataset, task, and baselines (2019, methodology): “The diarization baseline is based on the previously published Kaldi [32] recipe(7) for JHU’s submission to DIHARD I [14].”
End-to-End Neural Speaker Diarization with Permutation-Free Objectives (2019, methodology): “We compared the proposed method with the two conventional clustering-based systems [23].”
Speaker Recognition for Multi-speaker Conversations Using X-vectors (2019, methodology): “Mirroring progress in speaker recognition, recent systems have replaced i-vectors with DNN-based embeddings for capturing speaker characteristics [13, 14, 15].”
Neural Speaker Diarization with Speaker-Wise Chain Rule (2020, methodology): “We compared the proposed method with a conventional clustering-based system [7], which were created using the Kaldi CALLHOME diarization v2 recipe.”
A Review of Speaker Diarization: Recent Advances with Deep Learning (2021, methodology): “Therefore, after the advent of i-vector [51] and DNN-based embeddings [61] the segmentation based on speaker change point detection was mostly replaced by uniform segmentation [112, 113, 49], since varying length of the segment created an additional variability”