Evaluating speech features with the minimal-pair ABX task: analysis of the classical MFC/PLP pipeline

By Thomas Schatz, Vijayaditya Peddinti, Francis R. Bach, Aren Jansen, Hynek Hermansky, Emmanuel Dupoux. 2013

166 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

In Speech & Audio

In Signal Processing

Citations in the corpus

VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021, background): “(2020) to evaluate unsupervised speech representations by phoneme discriminability on 3 languages (English, French and Mandarin), and report ABX discriminability score (Schatz et al., 2013) on the 10s test”
Unsupervised Pretraining Transfers Well Across Languages (2020, methodology): “We consider the English, Mandarin and French benchmarks and we report the ABX score on them [24].”
Libri-Light: A Benchmark for ASR with Limited or No Supervision (2019, positive): “In this work, we substantially expand dataset size and use the same evaluation metrics (ABX [14]) for comparability.”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, background): “One of the core evaluations is the minimal-pair ABX task (Schatz et al., 2013), which aims to benchmark representations in terms of their discriminability between different sub-word speech units.”
Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks (2019, methodology): “As an intrinsic measure of discriminability, we use the ABX task [44].”
An Unsupervised Autoregressive Model for Speech Representation Learning (2019, background): “Others [1] have argued that there are many nuisance factors that might affect the performance of linear classifiers, and have proposed to use a contrastive loss for evaluation.”
Unsupervised Speech Representation Learning Using WaveNet Autoencoders (2019, methodology): “The ZeroSpeech evaluation scheme uses the minimal pair ABX test [56], [57] which assesses the model’s ability to”
Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner (2016, positive): “, 1994) AX/ABX discrimination (Carlin, Thomas, Jansen, & Hermansky, 2011; Schatz et al., 2013), cosine similarity (Landauer & Dumais, 1997)”
Encoding of phonology in a recurrent neural model of grounded speech (2017, methodology): “Here we evaluate the COCO Speech model on the Phoneme across Context (PaC) task of Schatz et al. (2013). This task consists of presenting a series of equal-length tuples (A,B,X) to the model, where A and B differ by one phoneme (either a vowel or a consonant), as do B and X , but A and X are not min”
A Brief Overview of Unsupervised Neural Speech Representation Learning (2022, background): “The longstanding zero resource speech challenge (ZeroSpeech) offers a new set of tasks for each edition (Versteegh et al. 2015; Dunbar et al. 2017, 2019, 2020, 2021) usually featuring a minimal-pair ABX task (Schatz et al. 2013, 2014).”
Learning Weakly Supervised Multimodal Phoneme Embeddings (2017, methodology): “To evaluate the phonetic discriminability of our embeddings, we use an ABX discrimination task [24, 25].”
A deep scattering spectrum — Deep Siamese network pipeline for unsupervised acoustic modeling (2016, methodology): “This evaluation is done by computing an ABX score [8, 9].”
Unsupervised Acoustic Unit Representation Learning for Voice Conversion using WaveNet Auto-encoders (2020, methodology): “The objective evaluation includes two metrics: Machine ABX [35] and bit-rate.”
Neural network vs. HMM speech recognition systems as models of human cross-linguistic phonetic perception (2018, methodology): “To quantify how easy it is to distinguish two phonetic categories based on representations produced by one of our models, we use a machine version of the ABX discrimination task (Schatz et al., 2013; Schatz, 2016).”
Parallel inference of dirichlet process Gaussian mixture models for unsupervised acoustic modeling: a feasibility study (2015, methodology): “Following the track 1 of the challenge, our evaluation metric is error rate in the ABX discriminability task [26, 27].”