Unsupervised spoken keyword spotting via segmental DTW on Gaussian posteriorgrams

By Yaodong Zhang, James R. Glass. 2009

376 citations recorded by the source corpus.

In this paper, we present an unsupervised learning framework to address the problem of detecting spoken keywords. Without any transcription information, a Gaussian Mixture Model is trained to label speech frames with a Gaussian posteriorgram. Given one or more spoken examples of a keyword, we use segmental dynamic time warping to compare the Gaussian posteriorgrams between keyword samples and test utterances. The keyword detection result is then obtained by ranking the distortion scores of all the test utterances. We examine the TIMIT corpus as a development set to tune the parameters in our system, and the MIT Lecture corpus for more substantial evaluation. The results demonstrate the viability and effectiveness of our unsupervised learning framework on the keyword spotting task.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised Speech Recognition (2021, methodology): “Speech data has been segmented and clustered in a large body of prior work (Varadarajan et al., 2008; Zhang and Glass, 2009; Gish et al., 2009; Lee and Glass, 2012; Lee et al., 2015; Ondel et al., 2016), including Bayesian methods (Kamper et al.”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, background): “A different line of work employs probabilistic graphical models to jointly segment and cluster the speech signal (Varadarajan et al., 2008; Zhang & Glass, 2009; Gish et al., 2009; Lee & Glass, 2012; Siu et al., 2014; Lee et al., 2015; Ondel et al., 2016; Kamper et al., 2016; 2017a).”
Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, background): “A central problem is ﬁnding frame-level feature representations that capture meaningful linguistic contrasts, such as phonetic categories, while being invariant to factors such as a speaker’s identity, gender or accent [2–5].”
Personalization of End-to-End Speech Recognition on Mobile Devices for Named Entities (2019, methodology): “In order to measure the performance based on a selected subset of words, we treat our problem as a retrieval task (similar to keyword spotting [26]).”
Learning Word-Like Units from Joint Audio-Visual Analysis (2017, background): “Many subsequent efforts extended these ideas(Jansen et al., 2010; Jansen & Van Durme, 2011; Dredze et al., 2010; Harwath et al., 2012; Zhang & Glass, 2009).”
Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation (2019, methodology): “When based on audio word2vec, QbE STD is much more efficient than conventional dynamic time warping (DTW) based approaches, because only the similarities between two single vectors are needed; this is in addition to the significantly better retrieval performance that it yields.”
Unsupervised Lexicon Discovery from Acoustic Input (2015, methodology): “…subsequences of the input that are similar in an acoustic feature space are identified, and, then clustered to discover categories corresponding to lexical items (Park and Glass, 2008; Zhang and Glass, 2009; Zhang et al., 2012; Jansen et al., 2010; Aimetti, 2009; McInnes and Goldwater, 2011).”
A Nonparametric Bayesian Approach to Acoustic Model Discovery (2012, methodology): “For each x i t , based on its hidden state, choose a mixture from the GMM of the chosen state.”
On the Contributions of Visual and Textual Supervision in Low-resource Semantic Speech Retrieval (2019, positive): “We measure performance using the following evaluation metrics, commonly used for retrieval tasks [25, 26].”
Fully unsupervised small-vocabulary speech recognition using a segmental Bayesian model (2015, background): “Recent studies in the speech processing community have applied unsupervised techniques to tasks such as phonetic discovery [1, 2, 3, 4], lexical discovery [5, 6, 7], spoken document retrieval [8] and query-by-example search [9, 10].”
EgoCom: A Multi-Person Multi-Modal Egocentric Communications Dataset (2020, background): “Turn-taking is also related to keyword-spotting (Wu et al., 2018; Zhang and Glass, 2009): words like “Okay" or “Well" may indicate ﬁnishing or starting speech.”
Extracting bottleneck features and word-like pairs from untranscribed speech for feature representation (2017, positive): “which usually involve unsupervised discovery of linguistic units in a target language, and these techniques have been shown successful in some downstream applications such as query-by-example spoken term detection (QbE-STD) [2, 3, 4, 5] and topic segmentation of spoken documents [6, 7, 8].”
Unsupervised word discovery from speech using automatic segmentation into syllable-like units (2015, result): “We also investigated agglomerative clustering of syllable tokens using DTW and observed very similar results to the uniform spectral slicing approach.”
Visually Grounded Learning of Keyword Prediction from Untranscribed Speech (2017, methodology): “For evaluation, we use three metrics [47, 48]: P@10 is the average precision (across keywords, in %) of the 10 highest-scoring proposals; P@N is the average precision of the top N proposals, with N the number of true occurrences of the keyword; and equal error rate (EER) is the average error rate at”
Parallel inference of dirichlet process Gaussian mixture models for unsupervised acoustic modeling: a feasibility study (2015, methodology): “Later on, GMM posteriorgrams, one kind of model-based posteriorgrams, was introduced to acoustic pattern discovery [2,16].”
A summary of the 2012 JHU CLSP workshop on zero resource speech technologies and models of early language acquisition (2013, methodology): “Given limited success of core recognition architectures in the zero resource setting, several alternative acoustic frontends and unsupervised acoustic models have been proposed in recen t years [2, 3, 4, 5, 1, 6, 7, 8, 9, 10], though there has been limited effo rt to evaluate these methods in a syst”
Spoken Content Retrieval—Beyond Cascading Speech Recognition with Text Retrieval (2015, methodology): “The segmental DTW was first used in unsupervised discovery of speech patterns from spoken documents [202], [203], but it can also be used here.”