Invariant Representations for Noisy Speech Recognition

By Dmitriy Serdyuk, Kartik Audhkhasi, Philemon Brakel, Bhuvana Ramabhadran, Samuel Thomas, Yoshua Bengio. 2016

70 citations recorded by the source corpus.

Modern automatic speech recognition (ASR) systems need to be robust under acoustic variability arising from environmental, speaker, channel, and recording conditions. Ensuring such robustness to variability is a challenge in modern day neural network-based ASR systems, especially when all types of variability are not seen during training. We attempt to address this problem by encouraging the neural network acoustic model to learn invariant feature representations. We use ideas from recent research on image generation using Generative Adversarial Networks and domain adaptation ideas extending adversarial gradient-based training. A recent work from Ganin et al. proposes to use adversarial training for image domain adaptation by using an intermediate representation from the main target classification network to deteriorate the domain classifier performance through a separate neural network. Our work focuses on investigating neural architectures which produce representations invariant to noise conditions for ASR. We evaluate the proposed architecture on the Aurora-4 task, a popular benchmark for noise robust ASR. We show that our method generalizes better than the standard multi-condition training especially when only a few noise categories are seen during training.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion? (2019, neutral): “While general form of acoustic variabilities have been studied [27], there is some work specifically on speaker invariance [28, 29].”
Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data (2017, background): “Learning to disentangle the mixed information from the surface representation is essential for a wide variety of speech applications: for example, noise robust speech recognition [41, 38, 37, 16], speaker verification [5], and voice conversion [40, 29, 24].”
Learning Independent Features with Adversarial Nets for Non-linear ICA (2017, background): “While most work on GANs applies this matching of distributions in the context of generative modelling, some recent work has extended these ideas to learn features which are invariant to different domains (Ganin et al., 2016) or noise conditions (Serdyuk et al., 2016).”
Speaker-Invariant Training Via Adversarial Learning (2018, methodology): “In speech, it has been applied to noiserobust [15, 16, 17, 18, 19] and conversational ASR [20] using gradient reversal layer [21] or domain separation network [22].”
Unsupervised Adaptation of Acoustic Models for ASR Using Utterance-Level Embeddings from Squeeze and Excitation Networks (2019, methodology): “Such networks have been used for achieving speaker invariance [24, 25, 26], noise invariance [27, 28], etc.”
Unsupervised adaptation with domain separation networks for robust speech recognition (2017, methodology): “This GRL approach has been applied to acoustic models for unsupervised adaptation in [25] and for increasing noise robustness in [26, 27].”
Learning Speaker Representations with Mutual Information (2018, background): “Beyond generative modeling, some works have extended this framework to learn features that are invariant to different domains [12] or to noise conditions [13].”
Adversarial target-invariant representation learning for domain generalization (2019, background): “Despite the success of domain adaptation algorithms in several tasks (Chattopadhyay et al., 2012; Serdyuk et al., 2016; Thompson et al., 2019), such techniques rely on the assumption that an unlabeled sample from a target distribution of interest is known at training time.”
Learning Noise-Invariant Representations for Robust Speech Recognition (2018, methodology): “The encoder meanwhile is trained both to minimize the classification loss and to fool the discriminator, in a scheme similar to the reverse gradient technique in the domainadversarial approach due to [38] and applied to speech by [27].”