Learning Filterbanks from Raw Speech for Phone Recognition

By Neil Zeghidour, Nicolas Usunier, Iasonas Kokkinos, Thomas Schatz, Gabriel Synnaeve, Emmanuel Dupoux. 2017

131 citations recorded by the source corpus.

We train a bank of complex filters that operates on the raw waveform and is fed into a convolutional neural network for end-to-end phone recognition. These time-domain filterbanks (TD-filterbanks) are initialized as an approximation of mel-filterbanks, and then fine-tuned jointly with the remaining convolutional architecture. We perform phone recognition experiments on TIMIT and show that for several architectures, models trained on TD- filterbanks consistently outperform their counterparts trained on comparable mel-filterbanks. We get our best performance by learning all front-end steps, from pre-emphasis up to averaging. Finally, we observe that the filters at convergence have an asymmetric impulse response, and that some of them remain almost analytic.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (2020, background): “dev PER test PER CNN + TD-filterbanks [59] 15.”
Effectiveness of self-supervised pre-training for speech recognition (2019, methodology): “tracted from the pre-trained BERT models is a character-based Wav2Letter setup of (Zeghidour et al., 2018) which uses seven consecutive blocks of convolutions (kernel size 5 with 1.”
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, methodology): “However, it does enable training BERT models which require a relatively small vocabulary. dev PER test PER CNN + TD-ﬁlterbanks (Zeghidour et al., 2018) 15.6 18.0 Li-GRU + fMLLR (Ravanelli et al., 2018) – 14.9 wav2vec (Schneider et al., 2019) 12.9 14.7 Baseline Next, we compare Gumbel-Softmax to…”
wav2vec: Unsupervised Pre-training for Speech Recognition (2019, neutral): “Alternatively, one could use other architectures such as the trainable frontend of Zeghidour et al. (2018a) amongst others.”
Fully Convolutional Speech Recognition (2018, methodology): “Several proposals to learn the front-end of speech recognition systems have been made [16, 17, 14, 18].”
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech (2020, background): “We also list the results of strong supervised systems [55]– [57].”
Speech Recognition (2018, unknown): “Alternatively, one could use other architectures such as the trainable frontend of Zeghidour et al. (2018a).”
Interpretable Convolutional Filters with SincNet (2018, methodology): “Similarly to our work, in [49] the convolutional filters are initialized with a predefined filter shape.”
Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps (2020, methodology): “Zeghidour et al. (2018) formulate a learnable version of a ﬁlter bank featurization; their ﬁlters are initialized as an approximation of MFSC features and then ﬁne-tuned jointly with the rest of the model.”