Deep Learning for Audio Signal Processing

By Hendrik Purwins, Bo Li, Tuomas Virtanen, Jan Schlüter, Shuo-yiin Chang, Tara N. Sainath. 2019

736 citations recorded by the source corpus.

Given the recent surge in developments of deep learning, this paper provides a review of the state-of-the-art deep learning techniques for audio signal processing. Speech, music, and environmental sound processing are considered side-by-side, in order to point out similarities and differences between the domains, highlighting general methods, problems, key references, and potential for cross fertilization between areas. The dominant feature representations (in particular, log-mel spectra and raw waveform) and deep learning models are reviewed, including convolutional neural networks, variants of the long short-term memory architecture, as well as more audio-specific neural network models. Subsequently, prominent deep learning application areas are covered, i.e., audio recognition (automatic speech recognition, music information retrieval, environmental sound detection, localization and tracking) and synthesis and transformation (source separation, audio enhancement, generative models for speech, sound, and music synthesis). Finally, key issues and future questions regarding deep learning applied to audio signal processing are identified.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Machine learning in acoustics: Theory and applications. (2019, background): “2,102 Since large amounts of training data might not be available, DL is not always practical.”
Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond (2021, background): “While deep learning has been effective in image detection [59], translation [60], speech recognition [61,62], sound synthesis [63] and even automated neural architecture search [64], clinical diagnosis and treatment tasks often need more care (e.g., patient interests, beliefs, social support and…”
Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges (2022, positive): “Inspired by this, some researchers further introduce the DL-enabled method in fields of computer vision (CV) [54], [55], [56], (natural language processing) NLP [57], [58], [59] and speech processing [60], [61], [62] into end-to-end communication system as SE approaches, which pioneered the modern S”
STanford EArthquake Dataset (STEAD): A Global Data Set of Seismic Signals for AI (2019, background): “In addition to these, similarity of seismic signals to other time series data such as audio (see [85]–[88]) suggests a potential for using STEAD beyond seismological applications.”
A Survey of Sound Source Localization with Deep Learning Methods (2021, neutral): “Nevertheless, only a short portion of each is dedicated to SSL with deep neural networks.”