Crepe: A Convolutional Representation for Pitch Estimation

By Jong Wook Kim, Justin Salamon, Peter Qi Li, Juan Pablo Bello. 2018

483 citations recorded by the source corpus.

The task of estimating the fundamental frequency of a monophonic sound recording, also known as pitch tracking, is fundamental to audio processing with multiple applications in speech processing and music information retrieval. To date, the best performing techniques, such as the pYIN algorithm, are based on a combination of DSP pipelines and heuristics. While such techniques perform very well on average, there remain many cases in which they fail to correctly estimate the pitch. In this paper, we propose a data-driven pitch tracking algorithm, CREPE, which is based on a deep convolutional neural network that operates directly on the time-domain waveform. We show that the proposed model produces state-of-the-art results, performing equally or better than pYIN. Furthermore, we evaluate the model's generalizability in terms of noise robustness. A pre-trained version of CREPE is made freely available as an open-source Python module for easy application.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

DDSP: Differentiable Digital Signal Processing (2020, methodology): “F0 outliers accounts for pitch tracking imper-fections in CREPE (Kim et al., 2018) vs. genuinely bad samples generated by the trained model.”
EdiTTS: Score-based Editing for Controllable Text-to-Speech (2021, methodology): “To further investigate the effectiveness of the pitch shifts created by EdiTTS, we extracted f0 values of the edited regions using CREPE [22].”
Improved Prosody from Learned F0 Codebook Representations for VQ-VAE Speech Waveform Reconstruction (2020, methodology): “F0 values were automatically extracted using a CREPE model [23] with a frame shift of 5ms.”
Textless Speech Emotion Conversion using Decomposed and Discrete Representations (2021, methodology): “We follow a similar approach to (Kim et al., 2018) and discretize the range of F0 values into d bins represented by one-hot encodings.”
Controllable Neural Prosody Synthesis (2020, methodology): “We use a modified version of CREPE [29] to extract ground truth F0 contours.”
Context-Aware Prosody Correction for Text-Based Speech Editing (2021, methodology): “Ground truth pitch values are extracted using a PyTorch port [20] of the Crepe pitch tracker [21].”
Unsupervised Cross-Domain Singing Voice Conversion (2020, methodology): “Adding the CREPE perceptual loss adds a signiﬁcant gain to the model performance across all metrics.”
NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis (2022, methodology): “For the baseline models, we chose 4 popular F 0 estimators, that is, praat, rapt, pyin, and crepe (Boersma et al., 1993; Boersma & Van Heuven, 2001; Jadoul et al., 2018; Talkin & Kleijn, 1995; Mauch & Dixon, 2014; Kim et al., 2018).”
Fast and Flexible Neural Audio Synthesis (2019, methodology): “F0 L 1 distance : Pitch tracking using CREPE, like with any pitch tracker, is not completely reliable.”
Chunked Autoregressive GAN for Conditional Waveform Synthesis (2021, methodology): “To align pitch features with the mel spectrogram, we replace the default padding of CREPE with the same reﬂection padding used for mel spectrogram computation (Section 5.2) and run CREPE with a hop size of 256 × 16000 / 22050 = 185 .”
Universal Speech Enhancement with Score-based Diffusion (2022, background): “These additional targets are pitch and harmonicity, as provided by crepe (Kim et al., 2018), voice activity detection (VAD) and loudness, provided by simple internal algorithms, and the deltas of all of them.”