Unsupervised Word Segmentation and Lexicon Discovery Using Acoustic Word Embeddings

By Herman Kamper, Aren Jansen, Sharon Goldwater. 2016

80 citations recorded by the source corpus.

In settings where only unlabeled speech data is available, speech technology needs to be developed without transcriptions, pronunciation dictionaries, or language modelling text. A similar problem is faced when modeling infant language acquisition. In these cases, categorical linguistic structure needs to be discovered directly from speech audio. We present a novel unsupervised Bayesian model that segments unlabeled speech and clusters the segments into hypothesized word groupings. The result is a complete unsupervised tokenization of the input speech in terms of discovered word types. In our approach, a potential word segment (of arbitrary length) is embedded in a fixed-dimensional acoustic vector space. The model, implemented as a Gibbs sampler, then builds a whole-word acoustic model in this space while jointly performing segmentation. We report word error rates in a small-vocabulary connected digit recognition task by mapping the unsupervised decoded output to ground truth transcriptions. The model achieves around 20% error rate, outperforming a previous HMM-based system by about 10% absolute. Moreover, in contrast to the baseline, our model does not require a pre-specified vocabulary size.

Subfields: Artificial Intelligence, Computational Linguistics & Semantics, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Linguistics & Semantics

In Speech & Audio

In Signal Processing

Citations in the corpus

Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, background): “A different line of work employs probabilistic graphical models to jointly segment and cluster the speech signal (Varadarajan et al., 2008; Zhang & Glass, 2009; Gish et al., 2009; Lee & Glass, 2012; Siu et al., 2014; Lee et al., 2015; Ondel et al., 2016; Kamper et al., 2016; 2017a).”
A segmental framework for fully-unsupervised large-vocabulary speech recognition (2016, methodology): “The latter, which we denote simply as WER, might leave some cluster unassigned and these are counted as errors [24].”
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces (2018, background): “This has prompted several studies on full-coverage approaches, where the entire speech input is segmented into word-like units [31, 32, 33, 34].”
An embedded segmental K-means model for unsupervised segmentation and clustering of speech (2017, methodology): “Full details are given in [20], but the Gibbs sampler looks very similar to Algorithm 1: the Bayesian GMM gives likelihood terms (“scores”) in order to find an optimal segmentation, while the segmentation hypothesizes boundaries for the word segments which are then clustered using the GMM.”
Large-Scale Representation Learning from Visually Grounded Untranscribed Speech (2019, background): “To address problems of this sort, there are likely many opportunities to combine ideas from unsupervised term discovery (Kamper et al., 2016; Bansal et al., 2017) with audiovisual word learning (Harwath et al., 2018) and models of visual grounding that have been applied to text (Kiros et al., 2018).”
Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation (2019, background): “1 Probabilistic Bayesian models are developed to construct a model which learns segmentation and representation (or clustering) jointly [21], [25], [64].”
Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech (2018, unknown): “Other approaches use Bayesian generative models at multiple levels of linguistic abstraction [13, 14, 15].”
Visually Grounded Learning of Keyword Prediction from Untranscribed Speech (2017, background): “Here we consider the problem of grounding un-labelled speech when paired with images.”
Multilingual bottle-neck feature learning from untranscribed speech (2017, unknown): “Recently there is an increasing research interest in unsupervised speech processing, which usually involves unsupervised discovery of linguistic units [1, 2, 3], and the derived techniques have been used in different applications, such as retrieval of spoken queries in a speech database [4, 5, 6] an”
Learning Word Embeddings: Unsupervised Methods for Fixed-size Representations of Variable-length Speech Segments (2018, methodology): “In speech, similar techniques could be very useful, and are already used for several applications such as query by example [7], spoken term discovery [8] and unsupervised representation learning [9, 10].”
Towards Bilingual Lexicon Discovery From Visually Grounded Speech Audio (2019, methodology): “Other works such as [10, 11, 12] used Bayesian generative approaches to cluster acoustic segments.”
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech (2017, unknown): “ery [39], and unsupervised segmentation and clustering [40], [41].”
Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings (2017, methodology): “This template-based embedding approach was subsequently used for unsupervised speech recognition in [18] and, more importantly, the full QbE system of [10], which we consider to be a baseline in our experiments.”