An embedded segmental K-means model for unsupervised segmentation and clustering of speech

By Herman Kamper, Karen Livescu, Sharon Goldwater. 2017

104 citations recorded by the source corpus.

Unsupervised segmentation and clustering of unlabelled speech are core problems in zero-resource speech processing. Most approaches lie at methodological extremes: some use probabilistic Bayesian models with convergence guarantees, while others opt for more efficient heuristic techniques. Despite competitive performance in previous work, the full Bayesian approach is difficult to scale to large speech corpora. We introduce an approximation to a recent Bayesian model that still has a clear objective function but improves efficiency by using hard clustering and segmentation rather than full Bayesian inference. Like its Bayesian counterpart, this embedded segmental K-means model (ES-KMeans) represents arbitrary-length word segments as fixed-dimensional acoustic word embeddings. We first compare ES-KMeans to previous approaches on common English and Xitsonga data sets (5 and 2.5 hours of speech): ES-KMeans outperforms a leading heuristic method in word segmentation, giving similar scores to the Bayesian model while being 5 times faster with fewer hyperparameters. However, its clusters are less pure than those of the other models. We then show that ES-KMeans scales to larger corpora by applying it to the 5 languages of the Zero Resource Speech Challenge 2017 (up to 45 hours), where it performs competitively compared to the challenge baseline.1

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised Speech Recognition (2021, methodology): “There has been a lot of prior work in unsupervised speech segmentation (Kamper et al., 2017a,b; Rasanen et al., 2015; Kreuk et al., 2020) but here we simply use a method based on clustering the wav2vec 2.0 speech representations c1, . . . , cT .”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, methodology): “A different line of work employs probabilistic graphical models to jointly segment and cluster the speech signal (Varadarajan et al., 2008; Zhang & Glass, 2009; Gish et al., 2009; Lee & Glass, 2012; Siu et al., 2014; Lee et al., 2015; Ondel et al., 2016; Kamper et al., 2016; 2017a).”
Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech (2018, background): “It would be interesting to explore less supervised seg-mentations to learn word boundaries [47, 48].”
UWSpeech: Speech to Speech Translation for Unwritten Languages (2020, background): “A variety of previous works [1, 6, 10, 11, 16, 25, 35, 44] have investigated the conversion between speech and their corresponding phonetic categories (discrete tokens) in an unsupervised manner, which mimics the way that human infants learn acoustic models in their mother tongue during their early…”
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces (2018, methodology): “The second one, called embedded segmental K-means model (ES-KMeans) [36], is an approximation to BES-GMM that uses hard clustering and segmentation, rather than full Bayesian inference.”
Self-Supervised Speech Representation Learning: A Review (2022, background): “For example, searching within a corpus of recorded speech for segments that match a given (written or spoken) query can be seen as finding segments whose representations are most similar to that of the query [118], [192], [193], [194]; word embeddings can be defined by pooling representations of ins”
Measuring the perceptual availability of phonological features during language acquisition using unsupervised binary stochastic autoencoders (2019, methodology): “Following preceding work showing improved unsupervised clustering when segments are given fixed-dimensional acoustic representations, thus abstracting away from the variable temporal dilation in natural speech (Kamper et al., 2017a,b), we resample all reconstruction targets to a length of 25 frames.”
Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching (2018, methodology): “Kamper et al. (2017) iterates between the clustering and segmentation steps.”
Learning Word Embeddings from Speech (2017, background): “In the future, it would be interesting to explore less supervised segmentations to learn word boundaries [Kamper et al., 2017b,a, 2015].”
Virtual Phone Discovery for Speech Synthesis Without Text (2019, methodology): “Embedded segmentalK-means (ES-KMeans) [28] jointly optimizes segmentation and clustering assignments.”