Speaker-Invariant Training Via Adversarial Learning
By Zhong Meng, Jinyu Li, Zhuo Chen, Yang Zhao, Vadim Mazalov, Yifan Gong, Biing-Hwang Fred Juang. 2018
115 citations recorded by the source corpus.
We propose a novel adversarial multi-task learning scheme, aiming at actively curtailing the inter-talker feature variability while maximizing its senone discriminability so as to enhance the performance of a deep neural network (DNN) based ASR system. We call the scheme speaker-invariant training (SIT). In SIT, a DNN acoustic model and a speaker classifier network are jointly optimized to minimize the senone (tied triphone state) classification loss, and simultaneously mini-maximize the speaker classification loss. A speaker-invariant and senone-discriminative deep feature is learned through this adversarial multi-task learning. With SIT, a canonical DNN acoustic model with significantly reduced variance in its output probabilities is learned with no explicit speaker-independent (SI) transformations or speaker-specific representations used in training or testing. Evaluated on the CHiME-3 dataset, the SIT achieves 4.99% relative word error rate (WER) improvement over the conventional SI acoustic model. With additional unsupervised speaker adaptation, the speaker-adapted (SA) SIT model achieves 4.86% relative WER gain over the SA SI acoustic model.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Signal Processing
Citations in the corpus
- Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion? (2019, neutral): “While general form of acoustic variabilities have been studied [27], there is some work specifically on speaker invariance [28, 29].”
- Cycle-Consistent Speech Enhancement (2018, methodology): “81% WER when evaluated on the clean acoustic model in [36].”
- To Reverse the Gradient or Not: an Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition (2018, methodology): “As suggested in [5, 11], we optimize the parameters using Stochastic Gradient Descent where we reverse the gradients of ∂Łspk/∂θr [5].”
- Recent progresses in deep learning based acoustic models (2017, background): “Recently, the concept of adversarial training [147] was explored for noise-robust ASR [148]–[155] and speaker invariant training [156], [157].”
- Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling (2019, background): “Motivated by successful applications of adversarial training [20] in a wide range of domain invariant learning tasks [21–24], this work proposes to add an auxiliary adversarial speaker classification task to explicitly target speaker-invariant feature learning.”
- Towards Unsupervised Learning of Speech Features in the Wild (2021, methodology): “Another technique consists in applying adversarial losses on a speaker classification auxiliary tasks to render the representation speaker invariant [20].”
- Unsupervised Adaptation of Acoustic Models for ASR Using Utterance-Level Embeddings from Squeeze and Excitation Networks (2019, methodology): “Such networks have been used for achieving speaker invariance [24, 25, 26], noise invariance [27, 28], etc.”
- Disentangling Correlated Speaker and Noise for Speech Synthesis via Data Augmentation and Adversarial Factorization (2019, methodology): “These two methods have been mainly applied for training robust discriminative models [18, 19, 12, 20, 21, 22], and are less studied in the context of building generative models.”
- Conditional Teacher-student Learning (2019, unknown): “[30, 31], subspace-based [32, 33] and adversarial learning-based [34, 35] approaches.”