On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration

By Eamonn J. Keogh, Shruti Kasetty. 2002

1,447 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

OpenML: networked science in machine learning (2014, background): “It has been shown that studies regularly contradict each other because they are biased toward diﬀerent datasets [20], or because they don’t take into account the eﬀects of dataset size, parameter optimization and feature selection [33; 17].”
Time series feature extraction for data mining using DWT and DFT (2003, background): “[17] tested many sophisticated distance functions on two benchmark datasets and found them not to be better than Euclidean distance.”
The UCR time series archive (2018, background): “As noted in a survey [2], during the first decade of research, the vast majority of papers tested only on a single artificial data set created by the proposing authors themselves [1], [3]–[5].”
Forecasting and operational research: a review (2008, methodology): “As in forecasting, the limitations of method evaluations on toy-datasets to derive best practices have been identified within DM (Keogh and Kasetty, 2003), with an increasing emphasis on ‘meta-learning’ (SmithMiles, 2008).”
OpenML Benchmarking Suites and the OpenML100 (2017, neutral): “This often results in suboptimal shortcuts in study designs, producing rather small-scale, one-off experiments that should be interpreted with caution (Aha, 1992), are hard to reproduce (Pedersen, 2008; Hirsh, 2008), and may even lead to contradictory results (Keogh and Kasetty, 2003).”
Highly Comparative Feature-Based Time-Series Classification (2014, background): “…this interdisciplinary wealth of scientiﬁc methods for understanding the properties and dynamics of time series has received less attention in the temporal data mining literature, which treats large databases of time series, typically with the aim of either clustering or classifying the data [1–3].”
On The Marriage of Lp-norms and Edit Distance (2004, methodology): “Compared to the standard CBF data [17], temporal shifting is introduced in the CBFtr data set.”
Experimental comparison of representation methods and distance measures for time series data (2010, methodology): “The myth of “DTW is too slow” seems to come from reading old literature, perhaps combined with implementation bias [34], and it perpetuates itself from paper to paper.”
A Bag-of-Features Framework to Classify Time Series (2013, methodology): “For time series, one nearest neighbor (NN) classifiers with euclidean (NNEuclidean), or a dynamic time warping distance (NNDTW) have been widely and successfully used [1], [2], [3], [4], [5].”
Semi-supervised time series classification (2006, background): “One-nearest-neighbor with Euclidean distance [22] 1.”
Indexing multi-dimensional time-series with support for multiple distance measures (2003, background): “In some datasets, there is no need to perform warping, since the Euclidean distance performs acceptably [11].”
Spatio-Temporal Data Mining (2017, background): “Another related area of research is time series data mining [Esling and Agon 2012; Keogh and Kasetty 2003; Liao 2005], where approaches for mining useful information from time-series databases have been explored.”
Complex network approaches to nonlinear time series analysis (2019, background): “Here, time series mining focuses more on indexing, clustering, classification, segmentation, motif discovery, and forecasting [9, 11].”
Time series shapelets: a novel technique that allows accurate, interpretable and fast classification (2010, background): “” Most other state-of-the-art time series/shape classifiers do not produce interpretable results (Ding et al. 2008; Keogh and Kasetty 2002).”
A review on time series data mining (2011, background): “Those researches have been studied in considerable detail by both database and pattern recognition communities for different domains of time series data (Keogh and Kasetty, 2002).”