A Sticky HDP-HMM With Application to Speaker Diarization

By Emily B. Fox, Erik B. Sudderth, Michael I. Jordan, Alan S. Willsky. 2009

416 citations recorded by the source corpus.

We consider the problem of speaker diarization, the problem of segmenting an audio recording of a meeting into temporal segments corresponding to individual speakers. The problem is rendered particularly difficult by the fact that we are not allowed to assume knowledge of the number of people participating in the meeting. To address this problem, we take a Bayesian nonparametric approach to speaker diarization that builds on the hierarchical Dirichlet process hidden Markov model (HDP-HMM) of Teh et al. [J. Amer. Statist. Assoc. 101 (2006) 1566--1581]. Although the basic HDP-HMM tends to over-segment the audio data---creating redundant states and rapidly switching among them---we describe an augmented HDP-HMM that provides effective control over the switching rate. We also show that this augmentation makes it possible to treat emission distributions nonparametrically. To scale the resulting architecture to realistic diarization problems, we develop a sampling algorithm that employs a truncated approximation of the Dirichlet process to jointly resample the full state sequence, greatly improving mixing rates. Working with a benchmark NIST data set, we show that our Bayesian nonparametric architecture yields state-of-the-art speaker diarization results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/82e2ac38df6c777c4ac9e7c93a42e1b531f4bbae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Bayesian Nonparametric Inference of Switching Dynamic Linear Models (2010, background): “Blackwell and MacQueen [33] introduced a Pólya urn representation of theθ′i: θ′i | θ ′ 1, . . . , θ ′ i−1 ∼ γ γ + i− 1 H + i−1 ∑ j=1 1 γ + i− 1 δθ′j = γ γ + i− 1 H + K ∑ k=1 nk γ + i− 1 δθk .”
Stochastic variational inference (2012, background): “…2000; Attias, 2000), latent Dirichlet allocation (Blei et al., 2003), hidden Markov models (and many variants) (Rabiner, 1989; Fine et al., 1998; Fox et al., 2011b; Paisley and Carin, 2009), Kalman ﬁl-ters (and many variants) (Kalman, 1960; Fox et al., 2011a), factorial models (Ghahramani and…”
A General Method for Robust Bayesian Modeling (2015, background): “Non-exchangeable models only contain global variables, such as time series models (Rabiner, 1989; Fine et al., 1998; Fox et al., 2011; Paisley and Carin, 2009) and models for network analysis (Airoldi, 2007; Airoldi et al.”
Discovering linguistic structures in speech: models and applications (2014, methodology): “Our model is inspired by previous applications of nonparametric Bayesian models to segmentation and clustering problems in natural language processing and speaker diarization [60, 45]; particularly, we adapt the inference method used in [60] to train our model.”
Bayesian semiparametric modeling of realized covariance matrices (2016, methodology): “A related model is discussed in Fox et al. (2011). Our version employs a hierarchical DP prior that links each of the DPM models in each state, in addition to a separate HDP that governs the rows of the transition matrix as before.”
A Nonparametric Bayesian Approach to Acoustic Model Discovery (2012, methodology): “Bayesian Model for Segmentation Our model is inspired by previous applications of nonparametric Bayesian models to segmentation problems in NLP and speaker diarization (Goldwater, 2009; Fox et al., 2011); particularly, we adapt the inference method used in (Goldwater, 2009) to our segmentation task.”
Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (2020, background): “Here β is the sticky parameter that increases the probability of self-transition to avoid rapidly switching between dynamics models [35].”
Unsupervised Incremental Online Learning and Prediction of Musical Audio Signals (2015, methodology): “The Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) [43] has been used for segmentation in conjunction with clustering.”