SDR – Half-baked or Well Done?

By Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, John R. Hershey. 2018

1,662 citations recorded by the source corpus.

In speech enhancement and source separation, signal-to-noise ratio is a ubiquitous objective measure of denoising/separation quality. A decade ago, the BSS_eval toolkit was developed to give researchers worldwide a way to evaluate the quality of their algorithms in a simple, fair, and hopefully insightful way: it attempted to account for channel variations, and to not only evaluate the total distortion in the estimated signal but also split it in terms of various factors such as remaining interference, newly added artifacts, and channel errors. In recent years, hundreds of papers have been relying on this toolkit to evaluate their proposed methods and compare them to previous works, often arguing that differences on the order of 0.1 dB proved the effectiveness of a method over others. We argue here that the signal-to-distortion ratio (SDR) implemented in the BSS_eval toolkit has generally been improperly used and abused, especially in the case of single-channel separation, resulting in misleading results. We propose to use a slightly modified definition, resulting in a simpler, more robust measure, called scale-invariant SDR (SI-SDR). We present various examples of critical failure of the original SDR that SI-SDR overcomes.

Subfields: Cognitive Neuroscience, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

  1. Deliang Wang
  2. Ke Tan
  3. John R. Hershey
  4. Yi Luo
  5. Jonathan Le Roux
  6. Nima Mesgarani
  7. Zhong-Qiu Wang
  8. Yuxuan Wang

In Speech & Audio

  1. Yossi Adi
  2. Xu Tan
  3. Lei He
  4. Sheng Zhao
  5. Jinyu Li
  6. Nima Mesgarani
  7. Wei-Ning Hsu
  8. Tao Qin

In Signal Processing

  1. Yossi Adi
  2. Jinyu Li
  3. Nima Mesgarani
  4. Deliang Wang
  5. Shinji Watanabe
  6. John R. Hershey
  7. Zhuo Chen
  8. Jonathan Le Roux

Citations in the corpus