Analysis of Audio-Visual Features for Unsupervised Speech Recognition

By Jennifer Drexler, James R. Glass. 2017

21 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Multimodal Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Multimodal Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, neutral): “…analysis of the linguistic abstractions, such as words and phones, which are learned by the models (Harwath & Glass, 2017; Harwath et al., 2018b; Drexler & Glass, 2017; Alishahi et al., 2017; Harwath et al., 2019; Harwath & Glass, 2019; Havard et al., 2019b), and the impact of jointly…”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “, 2018) Attention weights Word sense disambiguation Distribution measures (Drexler and Glass, 2017) Audio-visual CNN activations Phonemes, speakers, word identity Clustering, discrimination (Harwath and Glass, 2017) Audio-visual CNN embeddings Word classes Clustering”
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input (2018, background): “(2016) that semantic correspondences could be learned between images and speech waveforms at the signal level, with subsequent works providing evidence that linguistic units approximating phonemes and words are implicitly learned by these models (Alishahi et al. 2017; Chrupala et al. 2017; Drexler a”
Language learning using Speech to Image retrieval (2019, methodology): “[14, 29] have recently shown that the speech-to-image retrieval approach can be used to detect word boundaries and even discover sub-word units.”
Transfer Learning from Audio-Visual Grounding to Speech Recognition (2019, background): “In addition, along with the analysis in [27, 25], this work sheds light on using semantic level supervision to learn the compositional structure of a language.”
Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech (2018, background): “Other work has analyzed these models, and provided evidence that linguistic abstractions such as phones and words emerge in their internal representations [4, 5, 6, 7].”
Towards Visually Grounded Sub-word Speech Unit Discovery (2019, background): “While [21] and [22] examined the utility of the intermediate representations of visually-grounded speech models to perform tasks such as speaker, phoneme, and word discrimination, they did not investigate if and how discrete, sub-word units may be emerging within the models.”
Learning Words by Drawing Images (2019, background): “Other works have investigated the emergence of different kinds of linguistic phenomena, such as sub-word units and phonemes, within similar models [13, 2, 15, 23].”
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units (2020, background): “…2019), as well as how word-like and subword-like linguistic units can be made to emerge within these models (Har-wath and Glass, 2017; Harwath et al., 2019; Drexler and Glass, 2017; Alishahi et al., 2017; Harwath et al., 2019; Harwath and Glass, 2019; Havard et al., 2019b; Harwath et al., 2020).”