Unsupervised adaptation with domain separation networks for robust speech recognition

By Zhong Meng, Zhuo Chen, Vadim Mazalov, Jinyu Li, Yifan Gong. 2017

58 citations recorded by the source corpus.

Unsupervised domain adaptation of speech signal aims at adapting a well-trained source-domain acoustic model to the unlabeled data from target domain. This can be achieved by adversarial training of deep neural network (DNN) acoustic models to learn an intermediate deep representation that is both senone-discriminative and domain-invariant. Specifically, the DNN is trained to jointly optimize the primary task of senone classification and the secondary task of domain classification with adversarial objective functions. In this work, instead of only focusing on learning a domain-invariant feature (i.e. the shared component between domains), we also characterize the difference between the source and target domain distributions by explicitly modeling the private component of each domain through a private component extractor DNN. The private component is trained to be orthogonal with the shared component and thus implicitly increases the degree of domain-invariance of the shared component. A reconstructor DNN is used to reconstruct the original speech feature from the private and shared components as a regularization. This domain separation framework is applied to the unsupervised environment adaptation task and achieved 11.08% relative WER reduction from the gradient reversal layer training, a representative adversarial training method, for automatic speech recognition on CHiME-3 dataset.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Toward Domain-Invariant Speech Recognition via Large Scale Training (2018, methodology): “We also expect modeling strategies like domain adversarial training [18, 37, 38] and better training techniques [39] to improve performance.”
Semi-Supervised Speech Recognition via Local Prior Matching (2020, methodology): “Unlike the proposed method, previous studies focus on learning domain invariant features (Sun et al., 2017; Hsu & Glass, 2018; Meng et al., 2017; 2018; 2019) or data augmentation through learned speech transformations (Hsu et al., 2017; 2018).”
Cycle-Consistent Speech Enhancement (2018, methodology): “To improve the ASR performance, we enhance the 9137 noisy utterances in CHiME-3 with ACSE and re-train the clean DNNHMM acoustic model in [31].”
Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation (2019, methodology): “Similar ideas have been applied to domain adaptation for speech recognition [51]; while”
Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training (2020, background): “Note that DAT has been previously studied in speech recognition [17, 18, 19], speaker veriﬁcation [20, 21] as well as speech synthesis [15, 10] tasks with superior performance in learning noise-invariant features and attribute disentanglement.”
Conditional Teacher-student Learning (2019, background): “T/S learning [3, 8, 9] and adversarial learning [20, 21, 22, 23, 24] are two effective approaches that can suppress this domain mismatch by adapting a source-domain acoustic model to target-domain speech.”
Speaker-Invariant Training Via Adversarial Learning (2018, methodology): “In speech, it has been applied to noiserobust [15, 16, 17, 18, 19] and conversational ASR [20] using gradient reversal layer [21] or domain separation network [22].”