Full Bayesian Hidden Markov Model Variational Autoencoder for Acoustic Unit Discovery

By Thomas Glarner, Patrick Hanebrink, Janek Ebbers, Reinhold Häb-Umbach. 2018

16 citations recorded by the source corpus.

The invention of the Variational Autoencoder enables the application of Neural Networks to a wide range of tasks in unsupervised learning, including the field of Acoustic Unit Discovery (AUD). The recently proposed Hidden Markov Model Variational Autoencoder (HMMVAE) allows a joint training of a neural network based feature extractor and a structured prior for the latent space given by a Hidden Markov Model. It has been shown that the HMMVAE significantly outperforms pure GMM-HMM based systems on the AUD task. However, the HMMVAE cannot autonomously infer the number of acoustic units and thus relies on the GMM-HMM system for initialization. This paper introduces the Bayesian Hidden Markov Model Variational Autoencoder (BHMMVAE) which solves these issues by embedding the HMMVAE in a Bayesian framework with a Dirichlet Process Prior for the distribution of the acoustic units, and diagonal or full-covariance Gaussians as emission distributions. Experiments on TIMIT and Xitsonga show that the BHMMVAE is able to autonomously infer a reasonable number of acoustic units, can be initialized without supervision by a GMM-HMM system, achieves computationally efficient stochastic variational inference by using natural gradient descent, and, additionally, improves the AUD performance over the HMMVAE.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

In Speech & Audio

In Signal Processing

Citations in the corpus

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units (2021, neutral): “or sequential [28], [30], [32], [33], [43].”
Unsupervised Speech Representation Learning Using WaveNet Autoencoders (2019, methodology): “Introducing a HMM prior as in [73] could promote a latent representation which better mimics the temporal phonetic structure of speech.”
On Generative Spoken Language Modeling from Raw Audio (2021, neutral): “Various constraints can be imposed to the encoded space, such as temporal smoothness (Ebbers et al., 2017; Glarner et al., 2018; Khurana et al., 2019, 2020), discreteness (Ondel et al.”
Textless Speech Emotion Conversion using Decomposed and Discrete Representations (2021, background): “Various constraints can be imposed on the encoded space, such as temporal smoothness (Ebbers et al., 2017; Glarner et al., 2018; Khurana et al., 2020), discreteness (Ondel et al.”