Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition

By Tom Sercu, Vaibhava Goel. 2016

60 citations recorded by the source corpus.

In computer vision pixelwise dense prediction is the task of predicting a label for each pixel in the image. Convolutional neural networks achieve good performance on this task, while being computationally efficient. In this paper we carry these ideas over to the problem of assigning a sequence of labels to a set of speech frames, a task commonly known as framewise classification. We show that dense prediction view of framewise classification offers several advantages and insights, including computational efficiency and the ability to apply batch normalization. When doing dense prediction we pay specific attention to strided pooling in time and introduce an asymmetric dilated convolution, called time-dilated convolution, that allows for efficient and elegant implementation of pooling in time. We show results using time-dilated convolutions in a very deep VGG-style CNN with batch normalization on the Hub5 Switchboard-2000 benchmark task. With a big n-gram language model, we achieve 7.7% WER which is the best single model single-pass performance reported so far.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Li Deng
5. Bo Li
6. Yanmin Qian
7. Xiangnan He
8. Jianfei Cai

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Bo Li
5. Li Deng
6. Yanmin Qian
7. Yifan Gong
8. Xiangang Li

Citations in the corpus

• Recent advances in convolutional neural networks (2015, background): “This is different from the sliding window pooling in the previous deep networks, where the number of sliding windows depends on the input size.”
• Exploring neural transducers for end-to-end speech recognition (2017, methodology): “We don’t leverage any speaker information in our models, though it has been shown to reduce WER in previous works [27, 24].”
• Recent progresses in deep learning based acoustic models (2017, methodology): “Dilated CNN has outperformed other deep CNN models on the switchboard task [48].”
• Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review (2017, background): “…2015), and, in particular, dilated convolution (Yu & Koltun, 2015), which has shown promising results for diverse tasks such as speech recognition and synthesis (Sercu & Goel, 2016; Oord et al., 2016), machine translation (Kalchbrenner et al., 2016), and scene segmentation (Yu & Koltun, 2015).”
• Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling (2017, methodology): “Following the previous works (Zweig et al., 2016b; Povey et al., 2016; Xiong et al., 2016b; Sercu and Goel, 2016), evaluation is carried out on the NIST 2000 CTS test set, which comprises both Switch-board (SWB) and CallHome (CH) subsets.”
• Gated Residual Networks with Dilated Convolutions for Supervised Speech Separation (2018, unknown): “Motivated by recent research [10] [11] [12] on dilated convolutions, we propose a novel gated residual network (GRN) with dilated convolutions to deal with speech separation.”
• A Simple Convolutional Generative Network for Next Item Recommendation (2018, background): “It is worth noting that although the dilated convolution was invented for dense prediction in image generation tasks [4, 26, 31], and has been applied in other fields (e.g., acoustic [22, 26] and translation [18] tasks), it is yet unexplored in recommender systems with huge sparse data.”
• Deep Learning for Audio Signal Processing (2019, background): “be highly effective, but in order to be able to output an event activity vector at a sufficiently high temporal resolution, the degree of max pooling or stride over time should not be too large – if a large receptive field is desired, dilated convolution and dilated pooling can be used instead [114]”
• Single-Channel Multi-talker Speech Recognition with Permutation Invariant Training (2017, methodology): “These WERs are much higher than that on other corpora, such as Switchboard [38] on which the WER is now below 10.0% [18], [36], [39], [40]; 3) Although the close-talk data (AMI IHM) was used to generate mixed speech in this work, the existence of parallel far-field data (AMI SDM/MDM) allows us to…”