Analyzing Learned Representations of a Deep ASR Performance Prediction Model

By Zied Elloumi, Laurent Besacier, Olivier Galibert, Benjamin Lecouteux. 2018

14 citations recorded by the source corpus.

This paper addresses a relatively new task: prediction of ASR performance on unseen broadcast programs. In a previous paper, we presented an ASR performance prediction system using CNNs that encode both text (ASR transcript) and speech, in order to predict word error rate. This work is dedicated to the analysis of speech signal embeddings and text embeddings learnt by the CNN while training our prediction model. We try to better understand which information is captured by the deep model and its relation with different conditioning factors. It is shown that hidden layers convey a clear signal about speech style, accent and broadcast type. We then try to leverage these 3 types of information at training time through multi-task learning. Our experiments show that this allows to train slightly more efficient ASR performance prediction systems that - in addition - simultaneously tag the analyzed utterances according to their speech style, accent and broadcast program origin.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, background): “(Elloumi et al., 2018) CNN activations Style, accent, broadcast program Classification (Belinkov and Glass, 2017) CNN/RNN activations Phonetic units Classification (Dalvi et al.”
Designing and Interpreting Probes with Control Tasks (2019, methodology): “Applying dropout, the most popular probing regularization method (Adi et al., 2017; Belinkov and Glass, 2019; Şahin et al., 2019; Kim et al., 2019; Elloumi et al., 2018; Belinkov and Glass, 2017; Belinkov et al., 2018) does not consistently lead to high-accuracy, high-selectivity MLP probes across a”
Equity Beyond Bias in Language Technologies for Education (2019, background): “Sommerauer and Fokkens, 2018) and speech (Elloumi et al., 2018; Krug and Stober, 2018), reading comprehension (Kaushik and Lipton, 2018), and machine translation (Shi et al.”
Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations (2020, methodology): “, 2018) have been widely used to visualize the high dimensional activation-space of neural networks to identify the role of network areas in solving a given task (Liu et al., 2016; Rauber et al., 2016; Elloumi et al., 2018; V., 2019).”