Soft-DTW: a Differentiable Loss Function for Time-Series
By Marco Cuturi, Mathieu Blondel. International Conference on Machine Learning, 2017
863 citations recorded by the source corpus.
We propose in this paper a differentiable learning loss between time series, building upon the celebrated dynamic time warping (DTW) discrepancy. Unlike the Euclidean distance, DTW can compare time series of variable size and is robust to shifts or dilatations across the time dimension. To compute DTW, one typically solves a minimal-cost alignment problem between two time series using dynamic programming. Our work takes advantage of a smoothed formulation of DTW, called soft-DTW, that computes the soft-minimum of all alignment costs. We show in this paper that soft-DTW is a differentiable loss function, and that both its value and gradient can be computed with quadratic time/space complexity (DTW has quadratic time but linear space complexity). We show that this regularization is particularly well suited to average and cluster time series under the DTW geometry, a task for which our proposal significantly outperforms existing baselines. Next, we propose to tune the parameters of a machine that outputs time series by minimizing its fit with ground-truth labels in a soft-DTW sense.
Subfields: Artificial Intelligence, Signal Processing
View Paper: https://www.semanticscholar.org/paper/08a426042c9926419198ee22c9bf80e6e4b5791b
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Signal Processing
Citations in the corpus
- Disentangling Physical Dynamics From Unknown Factors for Unsupervised Video Prediction (2020, background): “Dedicated loss functions [10, 30] and Generative Adversarial Networks (GANs) have been investigated for sharper predictions [40, 62, 29].”
- Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling (2021, methodology): “Similarly, Parallel Tacotron 2 iteratively predicts the output and Soft-DTW is used to compute the loss for each predicted output.”
- InceptionTime: Finding AlexNet for time series classification (2019, neutral): “Another research area focused on proposing global alignment kernels such as SoftDTW introduced by Cuturi and Blondel (2017), that can be further used in a nearest centroid classification scheme.”
- Deep Learning for Audio Signal Processing (2019, background): “for the fact that slightly non-linearly warped signals sound similar, differentiable dynamic time warping distance [62] or earth mover’s distance such as in Wasserstein GANs [63] might be more suitable.”
- Tslearn, A Machine Learning Toolkit for Time Series Data (2020, background): “…in the example below and the corresponding Figure 1, where standard Euclidean k -means fails while DTW-based ones (Sakoe and Chiba, 1978; Petitjean et al., 2011; Cuturi and Blondel, 2017) can distinguish between different time series profiles: processing time series data to feed scikit-learn models.”
- D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation (2019, background): “Our D(3)TW is related to recent progress on continuous relaxation of discrete operations, including theorem proving [30], softmax function [16], logic programming [9], and dynamic programming [26, 5].”
- NaturalSpeech: End-to-End Text-to-Speech Synthesis With Human-Level Quality (2022, background): “…in (11): 1) Since the frame-level prior distribution p ( z (cid:5) | y ; θ pri ) cannot well align with the ground-truth speech frames due to the intrinsically inaccurate duration prediction in durator, we leverage a soft dynamic time warping (DTW) [49] version of KL loss for L bwd and L fwd .”
- End-to-End Adversarial Text-to-Speech (2020, positive): “We use a soft version of DTW instead [12], which replaces the minimum with the soft minimum: where τ is a temperature parameter which we set to 0 .”