Automatic Lyrics Transcription in Polyphonic Music: Does Background Music Help?

By Chitralekha Gupta, Emre Yilmaz, Haizhou Li. 2019

15 citations recorded by the source corpus.

Background music affects lyrics intelligibility of singing vocals in a music piece. Automatic lyrics transcription in polyphonic music is a challenging task because the singing vocals are corrupted by the background music. In this work, we propose to learn music genre-specific characteristics to train polyphonic acoustic models. For this purpose, we firstly study and compare several automatic speech recognition pipelines for the application of lyrics transcription. Later, we present the lyrics transcription performance of these music-informed acoustic models for the best-performing pipeline, and systematically study the impact of music genre and language model on the performance. With this genre-based approach, we explicitly model the characteristics of music, instead of trying to remove the background music as noise. The proposed approach achieves a significant improvement in performance in the lyrics transcription and alignment tasks on several well-known polyphonic test datasets, outperforming all comparable existing systems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Speech & Audio

In Signal Processing

Citations in the corpus

Jukebox: A Generative Model for Music (2020, methodology): “To address this, we use Spleeter (Hennequin et al., 2019) to extract vo-cals from each song and run NUS AutoLyricsAlign (Gupta et al., 2020) on the extracted vocals to obtain a word-level alignments of the lyrics, allowing us to more accurately provide the lyrics for a given chunk of audio.”