Variational Inference for Acoustic Unit Discovery

By Lucas Ondel, Lukáš Burget, Jan Honza Cernocký. 2016

96 citations recorded by the source corpus.

Abstract Recently, several nonparametric Bayesian models have been proposed to automatically discover acoustic units in unlabeled data. Most of them are trained using various versions of the Gibbs Sampling (GS) method. In this work, we consider Variational Bayes (VB) as alternative inference process. Even though VB yields an approximate solution of the posterior distribution it can be easily parallelized which makes it more suitable for large database. Results show that, notwithstanding VB inference is an order of magnitude faster, it outperforms GS in terms of accuracy.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

In Speech & Audio

In Signal Processing

Citations in the corpus

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units (2021, background): “More advanced systems can achieve better acoustic unit discovery performance using better graphical models [27], [28] or parameterizes the”
Unsupervised Speech Recognition (2021, methodology): “Speech data has been segmented and clustered in a large body of prior work (Varadarajan et al., 2008; Zhang and Glass, 2009; Gish et al., 2009; Lee and Glass, 2012; Lee et al., 2015; Ondel et al., 2016), including Bayesian methods (Kamper et al.”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, background): “A different line of work employs probabilistic graphical models to jointly segment and cluster the speech signal (Varadarajan et al., 2008; Zhang & Glass, 2009; Gish et al., 2009; Lee & Glass, 2012; Siu et al., 2014; Lee et al., 2015; Ondel et al., 2016; Kamper et al., 2016; 2017a).”
Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, methodology): “Here we consider discrete representation learning, also referred to as ‘acoustic unit dis-covery’ [12, 13] or ‘discrete subword learning’ [14].”
Unsupervised Speech Representation Learning Using WaveNet Autoencoders (2019, methodology): “Extending it with an HMM temporal structure for sub-phonetic units leads to the DP-HMM and the HDP-HMM [69], [70], [71].”
On Generative Spoken Language Modeling from Raw Audio (2021, neutral): “, 2019, 2020), discreteness (Ondel et al., 2016; van den Oord et al., 2017), and presence of hierarchy (Lee and Glass, 2012; Hsu et al.”
Generative Spoken Dialogue Language Modeling (2022, methodology): “Models trained with either autoencoder objectives (Ondel et al., 2016; van den Oord et al., 2017) or masked objectives (CPC: van den Oord et al., 2018; APC: Chung and Glass, 2020; wav2vec 2.0: Baevski et al., 2020; HuBERT: Hsu et al., 2021a; MockingJay: Liu et al., 2020) from raw speech can learn…”
The Zero Resource Speech Challenge 2019: TTS without T (2019, methodology): “This baseline uses out-of-the-box acoustic unit discovery [5] and an out-of-the-box speech synthesizer (Merlin, with the Ossian front end [6]).”
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input (2018, background): “Popular approaches are based on dynamic time warping (Jansen et al. 2010; Jansen and Van Durme 2011; Park and Glass 2008), or Bayesian generative models of the speech signal (Kamper et al. 2016; Lee and Glass 2012; Ondel et al. 2016).”
Data Augmenting Contrastive Learning of Speech Representations in the Time Domain (2020, methodology): “Clustering methods fit models like mixtures of Gaussians or HMM-GMMs and extract a discrete code or posteriograms [12, 13, 18, 19, 20].”
Learning Word-Like Units from Joint Audio-Visual Analysis (2017, methodology): “Alternative approaches based on Bayesian nonparametric modeling (Lee & Glass, 2012; Ondel et al., 2016) employed a generative model to cluster acoustic segments into phoneme-like categories, and related works aimed to segment and cluster either reference or learned phoneme-like tokens into…”
Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing (2021, background): “We then show that speaker normalization improves performance on two downstream tasks: acoustic unit discovery [13–16], and spoken language modeling [17,18].”
Large-Scale Representation Learning from Visually Grounded Untranscribed Speech (2019, background): “…explored various means to transform raw speech into symbolic forms with little or no supervision (Park and Glass, 2007; Varadara-jan et al., 2008; Ondel et al., 2016; Kamper et al., Figure 1: Models that encode speech segments and images into a shared latent space enable images to be retrieved…”
Self-Supervised Speech Representation Learning: A Review (2022, background): “Following the success of representation learning, baseline and challenge techniques have shifted from purely generative models [327], [328], deep autoencoders [83], [329], and incorporation of neural-network-”
Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge (2020, methodology): “The challenge baseline system combines a Dirichlet process Gaussian mixture model (DPGMM) for acoustic unit discovery [19] with a parametric speech synthesizer based on Merlin [39].”
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning (2020, background): “One of the long-standing goals of speech and cognitive scientists is to develop a computational model of language acquisition [1, 2, 3, 4].”
Linguistic Unit Discovery from Multi-Modal Inputs in Unwritten Languages: Summary of the “Speaking Rosetta” JSALT 2017 Workshop (2018, methodology): “Four types of acoustic features were compared: Mel-frequency Filterbanks (baseline, similar to [16] but with added speaker-dependent mean-variance normalization on the features before zero-padding/truncation), the pseudo-phones generated by the AUD system [3] (which were downsampled by a factor of 9”
Cyclic Spectral Modeling for Unsupervised Unit Discovery into Voice Conversion with Excitation and Waveform Modeling (2020, background): “A lot of work have been proposed for the unsupervised discovery of subword units [3, 6, 8, 11, 12, 13].”