Speaker Recognition from Raw Waveform with SincNet

By Mirco Ravanelli, Yoshua Bengio. Spoken Language Technology Workshop, 2018

875 citations recorded by the source corpus.

Deep learning is progressively gaining popularity as a viable alternative to i-vectors for speaker recognition. Promising results have been recently obtained with Convolutional Neural Networks (CNNs) when fed by raw speech samples directly. Rather than employing standard hand-crafted features, the latter CNNs learn low-level speech representations from waveforms, potentially allowing the network to better capture important narrow-band speaker characteristics such as pitch and formants. Proper design of the neural network is crucial to achieve this goal.This paper proposes a novel CNN architecture, called SincNet, that encourages the first convolutional layer to discover more meaningful filters. SincNet is based on parametrized sinc functions, which implement band-pass filters. In contrast to standard CNNs, that learn all elements of each filter, only low and high cutoff frequencies are directly learned from data with the proposed method. This offers a very compact and efficient way to derive a customized filter bank specifically tuned for the desired application.Our experiments, conducted on both speaker identification and speaker verification tasks, show that the proposed architecture converges faster and performs better than a standard CNN on raw waveforms.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/2540f32662aad59f66d0d9b00b9ad4721a9e522a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Speech Model Pre-training for End-to-End Spoken Language Understanding (2019, methodology): “The phoneme-level logits are com-5 puted using a linear classiﬁer: (1) The phoneme module is implemented using a SincNet layer [37,38], which processes the raw input waveform, followed by multiple convolutional layers and recurrent layers with pooling and dropout.”
End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network (2019, background): “Ravanelli and Bengio (Ravanelli & Bengio, 2018) proposed the SincNet, an end-to-end approach for speaker identiﬁcation and veriﬁcation.”
In defence of metric learning for speaker recognition (2020, methodology): “This weakness has been addressed by [11] who have proposed angular softmax (A-Softmax) where cosine similarity is used as logit input to the softmax layer, and a number of works have demonstrated its superiority over vanilla softmax in speaker recognition [6, 7, 8, 12, 13].”
Neural Harmonic-plus-Noise Waveform Model with Trainable Maximum Voice Frequency for Text-to-Speech Synthesis (2019, methodology): “Similarly, the high-pass ﬁlter coefﬁcients are deterministically computed by: Trainable sinc-based FIR ﬁlters have been used in SincNet [20].”
WaveletKernelNet: An Interpretable Deep Neural Network for Industrial Intelligent Diagnosis (2019, background): “Besides, according to the results of [29] that the first convolutional layer is one of the most critical parts of current waveform-based CNNs, it not only handles data with high dimensional but is also more susceptible to the problem of vanishing gradient, especially when the network architecture is”
End-to-End anti-spoofing with RawNet2 (2020, background): “One solution to these issues is SincNet [13, 14].”
Deep Learning for Audio Signal Processing (2019, background): “speaker recognition [96], language recognition [97] and speech translation [98].”
Filterbank Design for End-to-end Speech Separation (2019, positive): “In parallel, parameterized kernel-based filterbanks have been introduced as a front-end for speech and speaker recognition [15, 16].”
Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation (2022, neutral): “2-(a), AASIST uses a sinc convolutional layer based front-end [46].”
End-to-end speaker segmentation for overlap-aware resegmentation (2021, methodology): “The input sequence is passed to SincNet convolutional layers using the original configuration [19] – except for the stride of the very first layer which is set to 10 (so that SincNet frames are extracted every 16ms).”
St-Bert: Cross-Modal Language Model Pre-Training for End-to-End Spoken Language Understanding (2020, background): “[2] pre-train a SincNet-based neural network [12] with phonemes and words base ASR task and fine-tune it for intent classification after adding additional recurrent layers.”
Learning Speaker Representations with Mutual Information (2018, methodology): “This confirms the promising achievements obtained in [21, 22] in a standard supervised setting.”
Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps (2020, background): “(Ravanelli et al., 2018) 13 .”
Curriculum Learning: A Survey (2021, background): “Deep neural networks have become the state-of-the-art approach in a wide variety of tasks, ranging from object recognition in images [1], [2], [3], [4] and medical imaging [5], [6], [7], [8] to text classification [9], [10], [11], [12] and speech recognition [13], [14].”
Does Audio Deepfake Detection Generalize? (2022, background): “RawNet2 [23] is another end-to-end architecture and one of the ﬁrst to use Sinc-Layers [24], which correspond to rectangular band-pass ﬁlters, to extract information directly from raw waveforms.”
Interpretable Convolutional Filters with SincNet (2018, result): “Compared to standard CNNs, where the filter-bank characteristics depend on several parameters (each element of the filter vector is directly learned), SincNet convolves the waveform with a set of parametrized sinc functions that implement band-pass filters [38].”
Audio-Visual Model Distillation Using Acoustic Images (2019, background): “Several recent works address audio-related tasks such as natural sound recognition [26], speech separation and enhancement [1, 10], audio event classification or sound source localization [35, 42], either by directly modeling raw audio signals with 1D convolutions [5, 31, 37] or, most popularly, by ”
The Pytorch-kaldi Speech Recognition Toolkit (2018, methodology): “The SincNet model [33,34] is also implemented to perform speech recognition from raw waveform directly.”