Query-by-example spoken term detection using phonetic posteriorgram templates

By Timothy J. Hazen, Wade Shen, Christopher M. White. 2009 IEEE Workshop on Automatic Speech Recognition & Understanding, 2009

325 citations recorded by the source corpus.

This paper examines a query-by-example approach to spoken term detection in audio files. The approach is designed for low-resource situations in which limited or no in-domain training material is available and accurate word-based speech recognition capability is unavailable. Instead of using word or phone strings as search terms, the user presents the system with audio snippets of desired search terms to act as the queries. Query and test materials are represented using phonetic posteriorgrams obtained from a phonetic recognition system. Query matches in the test data are located using a modified dynamic time warping search between query templates and test utterances. Experiments using this approach are presented using data from the Fisher corpus.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/c5b17a4f2ab9b0af2549ff103371de91b3a3c661

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Streaming small-footprint keyword spotting using sequence-to-sequence models (2017, methodology): “There has also been some prior work which has explored building low-footprint KWS systems which can detect arbitrary keywords in the incoming speech: for example, using structured support vector machines [12, 13], and techniques based on matching incoming audio to example templates of the keyword (Q”
TTS Skins: Speaker Conversion via ASR (2019, methodology): “mploys an ASR encoder to extract Phonetic Posterior Grams (PPGs), to perform voice conversion without parallel data. PPGs is a vector of probabilities for each phonetic class for a speciﬁc time-frame [17]. Sun et al. [13] predict STRAIGHT vocoder features from PPG features, assumed to be speaker ind”
Unsupervised spoken keyword spotting via segmental DTW on Gaussian posteriorgrams (2009, methodology): “In contrast to the direct merging method used [16], we considered the reliability of each warping region on the test utterance.”
Visually grounded cross-lingual keyword spotting in speech (2018, background): “We also consider how a model ranks utterances in the test data from most to least relevant for each keyword [38, 39]: precision at ten (P@10) is the average precision of the ten highest-scoring proposals; precision at N (P@N ) is the average precision of the top N proposals, with N the number of tru”
Phonetic posteriorgrams for many-to-one voice conversion without parallel data training (2016, background): “A PPG is a time-versus-class matrix representing the posterior probabilities of each phonetic class for each specific time frame of one utterance [11, 12].”
Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings (2017, methodology): “Early approaches to low-resource QbE were based on performing DTW alignment of the query against a search collection either exactly [5, 6] or approximately [7–9].”
Viral Video Style: A Closer Look at Viral Videos on YouTube (2014, methodology): “DTW was originally proposed for template-based speech recognition [12] and has been widely used in speech utterance matching and spoken term detection [6].”
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech (2017, background): “…metrics are standard in (exact) keyword spotting, based on how a model ranks utterances in the test data from most to least relevant for each keyword (Hazen et al., 2009; Zhang and Glass, 2009): precision at ten (P@10) is the average precision of the ten highest-scoring proposals; precision at N…”
Automatic gain control and multi-style training for robust small-footprint keyword spotting with deep neural networks (2015, methodology): “Examples of alternative approaches to KWS, which avoid a full Viterbi decoding of the audio, include dynamic time warping (DTW) on phone posteriorgrams [5], training large-margin classifiers [6, 7] and point process models [8].”
Parallel inference of dirichlet process Gaussian mixture models for unsupervised acoustic modeling: a feasibility study (2015, methodology): “Noteworthily the posteriorgrams derived from phoneme recognizers and unsupervised subword models have also been used in low-resource applications [7, 8, 19].”
Code-Switched Speech Synthesis Using Bilingual Phonetic Posteriorgram with Only Monolingual Corpora (2020, background): “The PPG is a time-versus-class matrix representing the posterior probabilities of each phonetic class for a specific frame of one utterance [10, 11].”
Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversion (2020, background): “A PPG represents posterior probabilities of phonetic classes (phonemes or triphones/senones) for every frame of an utterance [23, 24].”
Spoken Content Retrieval—Beyond Cascading Speech Recognition with Text Retrieval (2015, methodology): “It was also proposed to use an multilayer perceptron (MLP) to transform the MFCCs into phonetic posteriors [208].”
Semantic Query-by-example Speech Search Using Visual Grounding (2019, methodology): “We use 3 metrics to quantify how well a QbE system predicts exact query matches [20, 21]: P @ 10 is the average precision (across keywords, in”
Visually Grounded Learning of Keyword Prediction from Untranscribed Speech (2017, methodology): “For evaluation, we use three metrics [47, 48]: P@10 is the average precision (across keywords, in %) of the 10 highest-scoring proposals; P@N is the average precision of the top N proposals, with N the number of true occurrences of the keyword; and equal error rate (EER) is the average error rate at”
Discovering linguistic structures in speech: models and applications (2014, methodology): “it seems to be [47 59] [41] [70 59 103] [67] [4] [99 77] 3”
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning (2020, background): “ognizer to generate intermediate phonetic representation [33], [34] as the inter-lingual between the speakers.”
On the Contributions of Visual and Textual Supervision in Low-resource Semantic Speech Retrieval (2019, unknown): “We measure performance using the following evaluation metrics, commonly used for retrieval tasks [25, 26].”