Time series classification from scratch with deep neural networks: A strong baseline

By Zhiguang Wang, Weizhong Yan, Tim Oates. 2016

2,081 citations recorded by the source corpus.

We propose a simple but strong baseline for time series classification from scratch with deep neural networks. Our proposed baseline models are pure end-to-end without any heavy preprocessing on the raw data or feature crafting. The proposed Fully Convolutional Network (FCN) achieves premium performance to other state-of-the-art approaches and our exploration of the very deep neural networks with the ResNet structure is also competitive. The global average pooling in our convolutional model enables the exploitation of the Class Activation Map (CAM) to find out the contributing region in the raw data for the specific labels. Our models provides a simple choice for the real world application and a good starting point for the future research. An overall analysis is provided to discuss the generalization capability of our models, learned features, network structures and the classification semantics.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Signal Processing

Citations in the corpus

Conditional Time Series Forecasting with Convolutional Neural Networks (2017, background): “A convolutional neural network (CNNs), see [18], is a biologically-inspired type of deep neural network (DNN) that has recently gained popularity due to its success in classiﬁcation problems (e.g. image recognition [16] or time series classiﬁcation [26]).”
Data-centric Artificial Intelligence: A Survey (2023, positive): “For instance, convolutional neural networks can be used in both images [120] and time series [236].”
Dilated convolutional neural networks for time series forecasting (2018, background): “A convolutional neural network (CNNs), see [18], is a biologically-inspired type of deep neural network (DNN) that has recently gained popularity due to its success in classiﬁcation problems (e.g. image recognition [16] or time series classiﬁcation [26]).”
The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances (2020, methodology): “Figure 18 shows the CAM of a network in InceptionTime for this problem case, highlighting the areas that lead to its prediction of Thursday.”
Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL (2020, background): “…neural networks : – standard : fully convolutional [25], Deep4Net [26] – resnet-based : one-dimensional adaptations of standard resnets [25], [27], wide resnets [28] and xresnets [29] – inception-based : InceptionTime [30] • recurrent neural networks : LSTM [31], GRU [32] • baseline…”
ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels (2019, background): “We compare Rocket to existing state-of-the-art methods for time series classi-ﬁcation, namely, BOSS, Shapelet Transform, Proximity Forest, ResNet, and HIVE-COTE.”
InceptionTime: Finding AlexNet for time series classification (2019, positive): “…of deep learning techniques in supervised learning such as image recognition (Zhang et al., 2018) and natural language processing (Guan et al., 2019), researchers started investigating these complex machine learning models for TSC (Wang et al., 2017; Cui et al., 2016; Ismail Fawaz et al., 2019a).”
Multivariate LSTM-FCNs for Time Series Classification (2018, methodology): “The fully convolutional block contains three temporal convolutional blocks, used as a feature extractor, which is replicated from the original fully convolutional block by Wang et al [34].”
Unsupervised Scalable Representation Learning for Multivariate Time Series (2019, methodology): “, 2017), but have recently been successfully used thanks to convolutional networks (Cui et al., 2016; Wang et al., 2017).”
MOMENT: A Family of Open Time-series Foundation Models (2024, methodology): “We compare MOMENT against 5 unsupervised representation learning methods (TS2Vec (Yue et al., 2022), TST (Zerveas et al., 2021), TS-TCC (Eldele et al., 2021), TNC (Tonekaboni et al., 2021), and T-Loss (Franceschi et al., 2019)), 8 supervised deep learning (CNN (Zebik et al., 2017), Encoder (Serr`a…”
Self-Attention for Raw Optical Satellite Time Series Classification (2019, methodology): “We compared the performance of the deep learning models LSTMRNN [61], Transformer [44], MS-ResNet [78], the DuPLO model [59], and TempCNN [69], as well as an RF classifier as shallow baseline.”
Time Series Data Augmentation for Deep Learning: A Survey (2020, methodology): “We train a fully convolutional network [Wang et al., 2017] to classify each time series in the training set.”
Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series (2018, methodology): “Meanwhile, temporal 1D-CNNs (TempCNNs) where convolutions are applied in the temporal domain have proven to be effective for handling the temporal dimension for time series classification [37], and 3D-CNN for both the spatial and temporal dimension in video classification [38].”
Wide and Deep Convolutional Neural Networks for Electricity-Theft Detection to Secure Smart Grids (2018, background): “This function can effectively prevent the overfitting [37].”
LSTM Fully Convolutional Networks for Time Series Classification (2017, methodology): “In this paper, the proposed model was evaluated using accuracy, rank based statistics, and the mean per class error as stated by Wang et al. [10].”
An empirical survey of data augmentation for time series classification with neural networks (2020, methodology): “[3] and is shown in Fig.”