Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks

By Federico Landini, Jan Profant, Mireia Díez, Lukáš Burget. 2020

261 citations recorded by the source corpus.

Abstract The recently proposed VBx diarization method uses a Bayesian hidden Markov model to find speaker clusters in a sequence of x-vectors. In this work we perform an extensive comparison of performance of the VBx diarization with other approaches in the literature and we show that VBx achieves superior performance on three of the most popular datasets for evaluating diarization: CALLHOME, AMI and DIHARD II datasets. Further, we present for the first time the derivation and update formulae for the VBx model, focusing on the efficiency and simplicity of this model as compared to the previous and more complex BHMM model working on frame-by-frame standard Cepstral features. Together with this publication, we release the recipe for training the x-vector extractors used in our experiments on both wide and narrowband data, and the VBx recipes that attain state-of-the-art performance on all three datasets. Besides, we point out the lack of a standardized evaluation protocol for AMI dataset and we propose a new protocol for both Beamformed and Mix-Headset audios based on the official AMI partitions and transcriptions.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

In Speech & Audio

In Signal Processing

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[128] Federico Landini, Ján Profant, Mireia Diez, and Lukáš Burget.”
Pyannote.audio 2.1 Speaker Diarization Pipeline: Principle, Benchmark, and Recipe (2023, methodology): “Compared to the standard approach that consists in extracting exactly one speaker embedding using a short (typically 1 or 2 seconds) periodic window [5], the proposed speaker embeddings are expected to be more reliable for two main reasons:”
End-to-end speaker segmentation for overlap-aware resegmentation (2021, methodology): “1 pretrained pipelines [16], dihard3 official baseline [9], and BUT’s VBx approach [15].”
A Review of Speaker Diarization: Recent Advances with Deep Learning (2021, methodology): “Since it is intractable to directory solve this problem, Variational Bayes method is used to estimate the model parameters that approximate P(Z,Y|X) [139, 141].”
SpeechBrain: A General-Purpose Speech Toolkit (2021, methodology): “The best diarization system available in SpeechBrain outperforms recent approaches based on meta-learning (MCGAN/ClusterGAN) [88], and Variational Bayes (VBx) [89] when the number of speakers is known (e.g., in a meeting).”
Encoder-Decoder Based Attractor Calculation for End-to-End Neural Diarization (2021, methodology): “We used the split and reference RTTMs provided in the VBx paper [30].”
Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors (2021, methodology): “As comparison methods, the state-of-the-art x-vector-based method (VBx) [2] and several EEND-based methods that can deal with a ﬂexible number of speakers [48, 10, 14, 27] were adopted.”
AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario (2021, methodology): “The dimension of speaker embedding is reduced from 256 to 128 for further clustering using the Variational Bayesian HMM clustering (VBx) [30] model.”
Powerset multi-class cross entropy loss for neural speaker diarization (2023, background): “Most dependable diarization approaches consist of a cascade of several steps [1]: voice activity detection to discard non-speech regions, speaker embedding to obtain discriminative speaker representations, and clustering to group speech segments by speaker identity.”