Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition

By Taesup Kim, Inchul Song, Yoshua Bengio. 2017

71 citations recorded by the source corpus.

Layer normalization is a recently introduced technique for normalizing the activities of neurons in deep neural networks to improve the training speed and stability. In this paper, we introduce a new layer normalization technique called Dynamic Layer Normalization (DLN) for adaptive neural acoustic modeling in speech recognition. By dynamically generating the scaling and shifting parameters in layer normalization, DLN adapts neural acoustic models to the acoustic variability arising from various factors such as speakers, channel noises, and environments. Unlike other adaptive acoustic models, our proposed approach does not require additional adaptation data or speaker information such as i-vectors. Moreover, the model size is fixed as it dynamically generates adaptation parameters. We apply our proposed DLN to deep bidirectional LSTM acoustic models and evaluate them on two benchmark datasets for large vocabulary ASR experiments: WSJ and TED-LIUM release 2. The experimental results show that our DLN improves neural acoustic models in terms of transcription accuracy by dynamically adapting to various speakers and environments.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Mohit Bansal
3. Quoc V. Le
4. Jianfeng Gao
5. Trevor Darrell
6. Percy Liang
7. Sergey Levine
8. Dacheng Tao

In Model Architectures

1. Serge J. Belongie
2. Aaron C. Courville
3. Felix Wu
4. Kilian Q. Weinberger
5. Boyi Li
6. Neil Houlsby
7. Vincent Dumoulin
8. Mario Lucic

In Speech & Audio

1. Aaron C. Courville
2. Sylvain Gelly
3. Felix Wu
4. Mario Lucic
5. Kilian Q. Weinberger

In Signal Processing

1. Sylvain Gelly
2. Mario Lucic
3. Ting-Li Chen
4. Harm de Vries
5. Aaron C. Courville
6. Boyi Li
7. Felix Wu
8. Neil Houlsby

Citations in the corpus

• FiLM: Visual Reasoning with a General Conditioning Layer (2017, background): “…Normalization, which has proven highly successful for image stylization (Dumoulin, Shlens, and Kudlur 2017; Ghiasi et al. 2017; Huang and Belongie 2017), speech recognition (Kim, Song, and Bengio 2017), and visual question answering (de Vries et al. 2017), demonstrating FiLM’s broad applicability.”
• On Self Modulation for Generative Adversarial Networks (2018, methodology): “Kim et al. (2017) use Dynamic Layer Normalization (Ba et al., 2016) for adaptive acoustic modelling.”
• Learning Visual Reasoning Without Strong Priors (2017, background): “CBN has proven highly effective for image stylization [14, 16], speech recognition [17], and traditional visual question answering tasks [15].”
• Positional Normalization (2019, methodology): “Inspired by Dynamic Layer Normalization and similar works [7, 28, 34, 36, 53], we propose a natural extension called Dynamic Moment Shortcut (DMS): instead of re-injecting µ and σ as is, we use a convolutional neural network that takes µ and σ as inputs to generate the β and γ for MS.”