Unsupervised Learning of Acoustic Sub-word Units

By Balakrishnan Varadarajan, Sanjeev Khudanpur, Emmanuel Dupoux. 2008

106 citations recorded by the source corpus.

Accurate unsupervised learning of phonemes of a language directly from speech is demonstrated via an algorithm for joint unsupervised learning of the topology and parameters of a hidden Markov model (HMM); states and short state-sequences through this HMM correspond to the learnt sub-word units. The algorithm, originally proposed for unsupervised learning of allophonic variations within a given phoneme set, has been adapted to learn without any knowledge of the phonemes. An evaluation methodology is also proposed, whereby the state-sequence that aligns to a test utterance is transduced in an automatic manner to a phoneme-sequence and compared to its manual transcription. Over 85% phoneme recognition accuracy is demonstrated for speaker-dependent learning from fluent, large-vocabulary speech.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised Speech Recognition (2021, methodology): “Speech data has been segmented and clustered in a large body of prior work (Varadarajan et al., 2008; Zhang and Glass, 2009; Gish et al., 2009; Lee and Glass, 2012; Lee et al., 2015; Ondel et al., 2016), including Bayesian methods (Kamper et al.”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, background): “A different line of work employs probabilistic graphical models to jointly segment and cluster the speech signal (Varadarajan et al., 2008; Zhang & Glass, 2009; Gish et al., 2009; Lee & Glass, 2012; Siu et al., 2014; Lee et al., 2015; Ondel et al., 2016; Kamper et al., 2016; 2017a).”
A segmental framework for fully-unsupervised large-vocabulary speech recognition (2016, methodology): “…(GMMs) to produce frame-level posteriorgrams [9,10], us-ing unsupervised hidden Markov models (HMMs) to obtain discrete categorical output in terms of discovered subword units [2,11,12], and using unsupervised neural networks (NNs) to obtain frame-level continuous vector representations [13–15].”
Data Augmenting Contrastive Learning of Speech Representations in the Time Domain (2020, methodology): “Clustering methods fit models like mixtures of Gaussians or HMM-GMMs and extract a discrete code or posteriograms [12, 13, 18, 19, 20].”
Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner (2016, positive): “For instance, Varadarajan, Khudanpur, and Dupoux (2008) have shown that a clustering algorithm based on Hidden Markov Models and Gaussian mixtures does not converge on phonetic segments, but rather, on much shorter (30 ms), highly context-sensitive acoustic events.”
An embedded segmental K-means model for unsupervised segmentation and clustering of speech (2017, background): “Several zero-resource tasks have been studied, including acoustic unit discovery [4–6], unsupervised representation learning [7–9], query-by-example search [10, 11] and topic modelling [12, 13].”
Discovering discrete subword units with binarized autoencoders and hidden-Markov-model encoders (2015, methodology): “For example, in [9] subwords are generated by iteratively splitting an initial single-state Hidden Markov Model with Gaussian output probabilities.”
Towards Visually Grounded Sub-word Speech Unit Discovery (2019, background): “Some proposed approaches operate at the word level [1, 2, 3], while a separate line of work is concerned with sub-word modeling [4, 5, 6, 7].”
A summary of the 2012 JHU CLSP workshop on zero resource speech technologies and models of early language acquisition (2013, background): “tures in the zero resource setting, several alternative acoustic frontends and unsupervised acoustic models have been proposed in recent years [2, 3, 4, 5, 1, 6, 7, 8, 9, 10], though there has been limited effort to evaluate these methods in a systematic way.”
Measuring the perceptual availability of phonological features during language acquisition using unsupervised binary stochastic autoencoders (2019, background): “…on unsupervised speech processing has inferred phonological structure from spoken utterances using either (1) discrete transition-based architectures (Varadarajan et al., 2008; Jansen and Church, 2011; Lee and Glass, 2012), which do attempt to discover featurally-related natural classes, or (2)…”
The zero resource speech challenge 2015 (2015, methodology): “Some approaches to this task use unsupervised clustering at the frame level using GMM’s [5, 2, 3].”
A Nonparametric Bayesian Approach to Acoustic Model Discovery (2012, methodology): “These two tasks are fair measuring methods, but performance on these tasks depends not only on the learned acoustic models, but also other components such as the label-to-phone transducer in (Varadarajan et al., 2008) and the graphone model in (Garcia and Gish, 2006).”
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech (2017, unknown): “Tasks include acoustic unit discovery [36], [37], query-by-example search [38], unsupervised term discov-”