Environmental Sound Recognition With Time–Frequency Audio Features

By Selina M. Chu, Shrikanth S. Narayanan, C.-C. Jay Kuo. IEEE Transactions on Audio, Speech, and Language Processing, 2009

707 citations recorded by the source corpus.

The paper considers the task of recognizing environmental sounds for the understanding of a scene or context surrounding an audio sensor. A variety of features have been proposed for audio recognition, including the popular Mel-frequency cepstral coefficients (MFCCs) which describe the audio spectral shape. Environmental sounds, such as chirpings of insects and sounds of rain which are typically noise-like with a broad flat spectrum, may include strong temporal domain signatures. However, only few temporal-domain features have been developed to characterize such diverse audio signals previously. Here, we perform an empirical feature analysis for audio environment characterization and propose to use the matching pursuit (MP) algorithm to obtain effective time-frequency features. The MP-based method utilizes a dictionary of atoms for feature selection, resulting in a flexible, intuitive and physically interpretable set of features. The MP-based feature is adopted to supplement the MFCC features to yield higher recognition accuracy for environmental sounds. Extensive experiments are conducted to demonstrate the effectiveness of these joint features for unstructured environmental sound classification, including listening tests to study human recognition capabilities. Our recognition system has shown to produce comparable performance as human listeners.

Subfields: Developmental Biology, Signal Processing

View Paper: https://www.semanticscholar.org/paper/cfbfef80faf1b72416c907a00380636ab7bc8d5d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Developmental Biology

In Signal Processing

Citations in the corpus

Acoustic Scene Classification: Classifying environments from the sounds they produce (2014, methodology): “[10] MFCCS, parametric (Gabor) GMMS Maximum likelihood”
Recurrent neural networks for polyphonic sound event detection in real life recordings (2016, methodology): “Applications of SED include for example acoustic surveillance [1], environmental context detection [2] and automatic audio indexing [3].”
Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification (2016, background): “Its applications range from context aware computing [1] and surveillance [2] to noise mitigation”
Unsupervised Filterbank Learning Using Convolutional Restricted Boltzmann Machine for Environmental Sound Classification (2017, methodology): “In the previous approaches, the ESC is typically conducted based on handcrafted features [7], such as, log-Mel features [8], matrix factorization [9, 10], dictionary learning [3], and wavelet-based features [11].”
Chime-home: A dataset for sound source recognition in a domestic environment (2015, neutral): “We motivate such a chunk size following [15, 19]; in [19] it is further observed that a 4-second chunk size yields 82% accuracy for human listeners distinguishing among 14 acoustic scene classes.”
End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network (2019, background): “Like typical automatic classiﬁcation systems, most of the approaches for environmental sound classiﬁcation rely on handcrafted features or learn representations from mid-level representations such as spectro-temporal features (Lude˜na-Choez & Gallardo-Antol´ın, 2016; Costa et al., 2012).”
Speech Emotion Recognition from Spectrograms with Deep Convolutional Neural Network (2017, background): “Besides hand-engineered features including prosodic features (pitch, energy, zero-crossings) [8-10], spectral features (linear predictor coefficients (LPC), linear predictor cepstral coefficients (LPCC), mel-frequency cepstral coefficients (MFCC) [11, 12], and non-linear features like Teager-energy-”
CNN-Based Learnable Gammatone Filterbank and Equal-Loudness Normalization for Environmental Sound Classification (2020, background): “Owing to its applicability to tasks such as audio content retrieval [2], acoustic surveillance (e.”
Detection and Classification of Acoustic Scenes and Events (2015, methodology): “However, single-label classiﬁcation is the design most commonly seen in prior literature in acoustic scene recognition [14]–[17], [10], and also lends itself to clear evaluation measures.”
A Dataset and Taxonomy for Urban Sound Research (2014, methodology): “See [5] for an example.”
Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks (2018, background): “Naturalists could use it for bio-diversity monitoring [14]–[16].”
Hierarchy-aware Loss Function on a Tree Structured Label Space for Audio Event Detection (2019, methodology): “[14] proposed a matching pursuit technique for finding useful time-frequency features to complement MFCC features, and obtained improved performance.”
Acoustic event detection in real life recordings (2010, background): “The work in [7] deals with direct audio context recognition.”
Indoor localization without infrastructure using the acoustic background spectrum (2011, methodology): “achieved similar performance using the matching pursuit (MP) signal decomposition [7].”
Metrics for Polyphonic Sound Event Detection (2016, background): “Systems performing sound event classiﬁcation are usually evaluated in terms of accuracy [2,4,5,11,13,14].”
Freesound Datasets: A Platform for the Creation of Open Audio Datasets (2017, background): “16 4s is taken as a reference length [30] since it was found to be enough for humans to recognize environmental sounds with 82% accuracy [4].”