Phoneme recognition using time-delay neural networks

By A. Waibel, Toshiyuki Hanazawa, Geoffrey E. Hinton, Kiyohiro Shikano, Kevin J. Lang. 1989

Subfields: Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Speech & Audio

Citations in the corpus

Convolutional Sequence to Sequence Learning (2017, background): “Convolutional neural networks are less common for sequence modeling, despite several advantages (Waibel et al., 1989; LeCun & Bengio, 1995).”
Character-level Convolutional Networks for Text Classification (2015, background): “In particular, time-delay networks used in the early days of deep learning research are essentially convolutional networks that model sequential data [1] [31].”
Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, methodology): “Recurrent neural networks and networks with convolution were also used in speech recognition around the same time [51, 67].”
Natural Language Processing (Almost) from Scratch (2011, methodology): “icular case, tagging a word requires the consideration of the whole sentence. When using neural networks, the natural choice to tackle this problem becomes a convolutional approach, rst introduced by Waibel et al. (1989) and also called Time Delay Neural Networks (TDNNs) in the literature. We descri”
Multi-Task Self-Supervised Learning for Robust Speech Recognition (2020, positive): “CHiME-5 results are based on Kaldi [37] and rely on a time-delay neural network (TDNN) [38, 39] acoustic model trained on PASE+ features.”
Fully Convolutional Speech Recognition (2018, methodology): “They have been first introduced as TDNNs to predict phoneme classes [10], and later to generate HMM posteriorgrams [11].”
Recent advances in convolutional neural networks (2015, background): “ations are replaced by densely connected layers for ASR tasks [320]. Yu et al . [321] propose a layer-wise context expansion with attention model for ASR. It is a variant of time-delay neural network [322] in which lower layers focus on extracting simple local patterns while higher layers exploit br”
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018, methodology): “They were used prominently for speech recognition in the 80s and 90s (Waibel et al., 1989; Bottou et al., 1990).”
Recent Trends in Deep Learning Based Natural Language Processing (2017, methodology): “To get a larger contextual range, the classic window approach is often coupled with a time-delay neural network (TDNN) (Waibel et al., 1989).”
Deep Learning: Methods and Applications (2014, methodology): “It is useful to point out that the time-delay neural network (TDNN) [202, 382] developed for early speech recognition is a special case and predecessor of the CNN when weight sharing is limited to one of the two dimensions, i.”
Simplifying Graph Convolutional Networks (2019, background): “Similarly, simple pre-defined linear image filters (Sobel & Feldman, 1968; Harris & Stephens, 1988) eventually gave rise to nonlinear CNNs with learned convolutional kernels (Waibel et al., 1989; LeCun et al., 1989).”
Machine learning of molecular electronic properties in chemical compound space (2013, methodology): “They have successfully been applied to different tasks such as object recognition [39] and speech recognition [40].”
Achieving Human Parity in Conversational Speech Recognition (2016, methodology): “While the basic structures have been well known for a long period [21, 22, 23, 24, 25, 26, 27], it is only recently that they have dominated the field as the best models for speech recognition.”
EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding (2015, background): “, time-delay [23, 24] or convolutional neural networks [25, 19], is straightforward to achieve.”
The microsoft 2016 conversational speech recognition system (2016, methodology): “The LACE model differs from the earlier TDNN models e.g. [3, 28] in the use of a learned attention mask and ResNet like linear pass-through.”
Deep Equilibrium Models (2019, background): “Alternatively, prior work has shown that deeply stacked temporal convolutions [49, 47, 17, 7] can achieve competitive results, especially on long sequences.”
Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks (2013, background): “In speech literature, CNN is referred to as time-delay neural network [3].”
Image Segmentation Using Deep Learning: A Survey (2020, background): “[17] introduced CNNs with weights shared among temporal receptive fields and backpropagation training for phoneme recognition, and LeCun et al.”