Deep maxout networks for low-resource speech recognition

By Yajie Miao, Florian Metze, Shourabh Rawat. 2013

112 citations recorded by the source corpus.

As a feed-forward architecture, the recently proposed maxout networks integrate dropout naturally and show state-of-the-art results on various computer vision datasets. This paper investigates the application of deep maxout networks (DMNs) to large vocabulary continuous speech recognition (LVCSR) tasks. Our focus is on the particular advantage of DMNs under low-resource conditions with limited transcribed speech. We extend DMNs to hybrid and bottleneck feature systems, and explore optimal network structures (number of maxout layers, pooling strategy, etc) for both setups. On the newly released Babel corpus, behaviors of DMNs are extensively studied under different levels of data availability. Experiments show that DMNs improve low-resource speech recognition significantly. Moreover, DMNs introduce sparsity to their hidden activations and thus can act as sparse feature extractors.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Deep Learning: Methods and Applications (2014, methodology): “Another new type of DNN units demonstrated more recently to be useful for speech recognition is the “maxout” units, which were used for forming the deep maxout network as described in [244].”
Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks (2016, background): “After these successes on phoneme recognition, similar systems have been proposed in which multiple layers of RNNs were combined with CTC to perform large vocabulary continuous speech recognition [7, 16].”
Improving deep neural network acoustic models using generalized maxout networks (2014, background): “Maxout networks, combined with “dropout” [7], have given state-of-the-art performance in various computer vision tasks [6], and have also achieved improvements in speech recognition tasks [8, 9].”
Speaker Adaptive Training of Deep Neural Network Acoustic Models Using I-Vectors (2015, unknown): “, rectifier [34] and maxout [35], [36]).”
Convolutional deep maxout networks for phone recognition (2014, background): “than ReLU networks, and the biggest gains were reported under low-resource conditions [10, 11, 12].”
Self-Paced Curriculum Learning (2015, background): “…(Tan, Jiang, and Neo 2014; Liu et al. 2013; Zhang et al. 2014; Inoue and Shinoda 2014; Jiang, Hauptmann, and Xiang 2012; Tang et al. 2012; Yu, Jiang, and Hauptmann 2014; Cao et al. 2013), and by automatic speech recognition (Miao, Metze, and Rawat 2013; Miao et al. 2014; Chiu and Rudnicky 2013).”
On speaker adaptation of long short-term memory recurrent neural networks (2015, background): “On a wide range of large vocabulary continuous speech recognition (LVCSR) tasks, deep neural networks (DNNs) have shown better performance than the traditional Gaussian mixture models (GMMs) [1, 2, 3].”