Segmental acoustic indexing for zero resource keyword search

By Keith D. Levin, Aren Jansen, Benjamin Van Durme. 2015

77 citations recorded by the source corpus.

The task of zero resource query-by-example keyword search has received much attention in recent years as the speech technology needs of the developing world grow. These systems traditionally rely upon dynamic time warping (DTW) based retrieval algorithms with runtimes that are linear in the size of the search collection. As a result, their scalability substantially lags that of their supervised counterparts, which take advantage of efficient word-based indices. In this paper, we present a novel audio indexing approach called Segmental Randomized Acoustic Indexing and Logarithmic-time Search (S-RAILS). S-RAILS generalizes the original frame-based RAILS methodology to word-scale segments by exploiting a recently proposed acoustic segment embedding technique. By indexing word-scale segments directly, we avoid higher cost frame-based processing of RAILS while taking advantage of the improved lexical discrimination of the embeddings. Using the same conversational telephone speech benchmark, we demonstrate major improvements in both speed and accuracy over the original RAILS system.

Subfields: Artificial Intelligence, Information Retrieval & Search, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, background): “From a technology perspective, such features could be used in downstream speech applications requiring symbolic or sparse input, e.g., for faster retrieval in speech search systems [19, 20].”
A segmental framework for fully-unsupervised large-vocabulary speech recognition (2016, background): “This is important in tasks such as query-by-example search [16, 17], where a system needs to ﬁnd all the utterances in a corpus containing a spoken query, or in unsupervised term discovery (UTD), where a system needs to automatically ﬁnd repeated word-or phrase-like patterns in a speech collection…”
Audio Word2Vec: Unsupervised Learning of Audio Segment Representations Using Sequence-to-Sequence Autoencoder (2016, methodology): “This approach is inspired from the previous work [10], but completely different in the ways to represent the audio segments.”
An embedded segmental K-means model for unsupervised segmentation and clustering of speech (2017, methodology): “Here we instead follow an acoustic word embedding approach [11,32,33]: an embedding function fe is used to map a variable length speech segment to a single embedding vector x ∈ R in a fixed-dimensional space, i.”
Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings (2017, methodology): “In several of these studies, neural approaches are shown to far outperform templatebased embeddings (such as those used in [10]) on an isolatedword discrimination task, which can be viewed as a proxy for QbE.”
Multi-view Recurrent Neural Acoustic Word Embeddings (2016, methodology): “This approach produced improved performance on a word discrimination task compared to using raw DTW distances, and was later also applied successfully for a query-by-example task (Levin et al., 2015).”
Deep convolutional acoustic word embeddings using word-pair side information (2015, methodology): “Several approaches were developed in [3], and in [14] these were successfully applied in a query-by-example search system.”
Self-Supervised Speech Representation Learning: A Review (2022, background): “For example, searching within a corpus of recorded speech for segments that match a given (written or spoken) query can be seen as finding segments whose representations are most similar to that of the query [118], [192], [193], [194]; word embeddings can be defined by pooling representations of ins”
Fully unsupervised small-vocabulary speech recognition using a segmental Bayesian model (2015, background): “This approach has found success in other unsupervised studies [19, 20, 21] since search and clustering tasks are more efficient in a fixeddimensional space.”
Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Models (2019, background): “Substantial effort was made to learn signal representations directly from speech signals in an unsupervised way, which could be a good step towards unsupervised ASR, and has been shown useful in various tasks, such as speaker identification [3], spoken term detection [4, 5, 6], and spoken document r”
Unsupervised Word Segmentation and Lexicon Discovery Using Acoustic Word Embeddings (2016, background): “Examples include the query-by-example systems of [1]–[4], and the unsupervised term discovery (UTD) systems of [5], [6], which aim to find repeated words or phrases in a”
Learning Weakly Supervised Multimodal Phoneme Embeddings (2017, methodology): “In particular, we will use a Siamese DNN architecture which feeds on word-level side information (the fact that two words are the same or different: [5, 6, 7, 8]), which demonstrably can be discovered automatically from continuous speech using spo-Figure ken term discovery ([7, 9]).”
Multilingual bottle-neck feature learning from untranscribed speech (2017, unknown): “Recently there is an increasing research interest in unsupervised speech processing, which usually involves unsupervised discovery of linguistic units [1, 2, 3], and the derived techniques have been used in different applications, such as retrieval of spoken queries in a speech database [4, 5, 6] an”
Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation (2019, background): “represent audio segments as fixed-length vectors [30], [31].”
Almost-unsupervised Speech Recognition with Close-to-zero Resource Based on Phonetic Structures Learned from Very Small Unpaired Speech and Text Data (2018, background): “There exist many approaches which can segment utterances automatically [14, a r X i v : 1810 .”
Semantic Query-by-example Speech Search Using Visual Grounding (2019, methodology): “In this respect our approach is similar to those in recent embedding-based QbE work [14–17], and also some embedding-based spoken term detection work [18].”
Discriminative acoustic word embeddings: Tecurrent neural network-based approaches (2016, methodology): “Therehas been some, thus far limited, work on acoustic word embeddings, focused on a number of embedding models, training approaches, and tasks [10, 11, 12, 13, 14, 15, 16, 17].”
Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings (2018, background): “For spoken document retrieval, by representing audio segments as vectors, search can be much more efﬁcient compared to template matching over audio frames [12, 18, 19].”