Learning the speech front-end with raw waveform CLDNNs

By Tara N. Sainath, Ron J. Weiss, Andrew W. Senior, Kevin Wilson, Oriol Vinyals. 2015

526 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

In Signal Processing

Citations in the corpus

WaveNet: A Generative Model for Raw Audio (2016, methodology): “Traditionally, speech recognition research has largely focused on using log mel-filterbank energies or mel-frequency cepstral coefficients (MFCCs), but has been moving to raw audio recently (Palaz et al., 2013; Tüske et al., 2014; Hoshen et al., 2015; Sainath et al., 2015).”
Learning Filterbanks from Raw Speech for Phone Recognition (2017, background): “[6, 7] propose an architecture composed of a convolutional layer followed by max-pooling and a nonlinearity, so that gammatone filterbanks correspond to a particular configuration of the network.”
Look, Listen and Learn (2017, background): “, rather than a single video frame, but it would be interesting to explore what more is gained from this stronger condition. Also, we would like to transition to using raw waveforms over spectrograms [2,28,32]. Additionally, it would be interesting to learn from the recently released large dataset o”
Fully Convolutional Speech Recognition (2018, background): “Several proposals to learn the front-end of speech recognition systems have been made [16, 17, 14, 18].”
Dilated Recurrent Neural Networks (2017, methodology): “Recently the CLDNN family of models [19] managed to match or surpass the log mel-frequency features in several speech problems using waveform.”
VoxCeleb: A Large-Scale Speaker Identification Dataset (2017, background): “This has led to a very recent shift from handcrafted features to the domain of deep CNNs which can be applied to higher dimensional inputs [13, 14] and for speaker identification [15].”
Adversarial Audio Synthesis (2018, background): “While a minority of recent research in discriminative audio classiﬁcation tasks has used raw audio input (Sainath et al., 2015; Lee et al., 2017), most of these approaches operate on spectrogram representations of audio.”
Specaugment on Large Scale Datasets (2019, methodology): “MTR has been successfully applied to HMM-based systems [18, 19] and end-to-end LAS models [5, 20, 21].”
Deep Speaker: an End-to-End Neural Speaker Embedding System (2017, methodology): “We also investigate stacked gated recurrent unit (GRU) [17] layers as an alternative for frame-level feature extraction, since they have proven to be effective for speech processing applications [18][19].”
TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation (2017, methodology): “In recent years, this approach has been used successfully applied in tasks such as speech recognition, speech synthesis and speech enhancement [9, 10, 11, 12, 13], but waveform-level speech separation with deep learning has not been investigated yet.”
Between-Class Learning for Image Classification (2017, background): “In fact, recent studies have demonstrated that CNNs can learn speech and sounds directly from raw waveforms, and each filter learns to respond to a particular frequency area [21, 26, 7].”
Deep Convolutional Neural Networks with Layer-Wise Context Expansion and Attention (2016, methodology): “Various CNN structures [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24] have been evaluated for speech recognition.”
Speaker Recognition from Raw Waveform with SincNet (2018, background): “This approach has shown promise in speech [22–26], including emotion tasks [27], speaker recognition [32], spoofing detection [31], and speech synthesis [28, 29].”
Feature Learning with Raw-Waveform CLDNNs for Voice Activity Detection (2016, methodology): “This result is in contrast with our findings in [14] where using a similar architecture for acoustic modeling resulted in the learned filterbanks devoted more filters to lower frequencies.”
RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification (2019, methodology): “Direct modeling of raw waveforms using deep neural networks (DNNs) is increasingly prevalent in a number of tasks due to advances in deep learning [1–8].”