Embedding-Based Speaker Adaptive Training of Deep Neural Networks

By Xiaodong Cui, Vaibhava Goel, George Saon. 2017

41 citations recorded by the source corpus.

An embedding-based speaker adaptive training (SAT) approach is proposed and investigated in this paper for deep neural network acoustic modeling. In this approach, speaker embedding vectors, which are a constant given a particular speaker, are mapped through a control network to layer-dependent element-wise affine transformations to canonicalize the internal feature representations at the output of hidden layers of a main network. The control network for generating the speaker-dependent mappings is jointly estimated with the main network for the overall speaker adaptive acoustic modeling. Experiments on large vocabulary continuous speech recognition (LVCSR) tasks show that the proposed SAT scheme can yield superior performance over the widely-used speaker-aware training using i-vectors with speaker-adapted input features.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Neural Voice Cloning with a Few Samples (2018, methodology): “The ﬁrst group is speaker adaptation for the whole-model [Yu et al., 2013], a portion of the model [Miao and Metze, 2015, Cui et al., 2017], or merely to a speaker embedding [Abdel-Hamid and Jiang, 2013, Xue et al., 2014].”
Unsupervised Adaptation of Acoustic Models for ASR Using Utterance-Level Embeddings from Squeeze and Excitation Networks (2019, methodology): “In [32], i-vectors and fMLLR features-based DNN speaker embeddings were used to generate conditional scaling and biasing parameters.”
Towards Unsupervised Learning of Speech Features in the Wild (2021, methodology): “Some models introduce speaker embeddings as auxiliary inputs [16, 17], other apply speaker-aware layer-wise affine transformation [18], or a speaker memory [19] .”
Ultra-Low Precision 4-bit Training of Deep Neural Networks (2020, methodology): “In speech, TPR FP4 shows ~1.4% degradation in Word Error Rate (WER) when training a 4-layer Bidirectional LSTM model on the SWB300 dataset [40], and 0.5% WER loss when we use FP8 for dw (update GEMM) computations.”
Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks (2019, background): “This scheme, however, results in > 4% accuracy degradation on ResNet18/ImageNet [24].”