Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings

By Shane Settle, Keith D. Levin, Herman Kamper, Karen Livescu. 2017

91 citations recorded by the source corpus.

Query-by-example search often uses dynamic time warping (DTW) for comparing queries and proposed matching segments. Recent work has shown that comparing speech segments by representing them as fixed-dimensional vectors --- acoustic word embeddings --- and measuring their vector distance (e.g., cosine distance) can discriminate between words more accurately than DTW-based approaches. We consider an approach to query-by-example search that embeds both the query and database segments according to a neural model, followed by nearest-neighbor search to find the matching segments. Earlier work on embedding-based query-by-example, using template-based acoustic word embeddings, achieved competitive performance. We find that our embeddings, based on recurrent neural networks trained to optimize word discrimination, achieve substantial improvements in performance and run-time efficiency over the previous approaches.

Subfields: Artificial Intelligence, Information Retrieval & Search, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Speech & Audio

In Signal Processing

Citations in the corpus

Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, background): “, for faster retrieval in speech search systems [19, 20].”
Learning Word Embeddings: Unsupervised Methods for Fixed-size Representations of Variable-length Speech Segments (2018, methodology): “In speech, similar techniques could be very useful, and are already used for several applications such as query by example [7], spoken term discovery [8] and unsupervised representation learning [9, 10].”
Self-Supervised Speech Representation Learning: A Review (2022, background): “For example, searching within a corpus of recorded speech for segments that match a given (written or spoken) query can be seen as finding segments whose representations are most similar to that of the query [118], [192], [193], [194]; word embeddings can be defined by pooling representations of ins”
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech (2017, background): “…models that, instead of using exact transcriptions, can learn from weaker forms of supervision, e.g., known word pairs (Syn-naeve et al., 2014a; Settle et al., 2017), translation text (Duong et al., 2016; Bansal et al., 2017; Weiss et al., 2017), or unordered word labels (Palaz et al., 2016).”
Semantic Query-by-example Speech Search Using Visual Grounding (2019, methodology): “Various acoustic embedding functions have been proposed, with neural models used in several studies [16, 18, 23–30].”