Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures

By T. Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, Jian Li. arXiv.org, 2022

294 citations recorded by the source corpus.

Multivariate time series forecasting has seen widely ranging applications in various domains, including finance, traffic, energy, and healthcare. To capture the sophisticated temporal patterns, plenty of research studies designed complex neural network architectures based on many variants of RNNs, GNNs, and Transformers. However, complex models are often computationally expensive and thus face a severe challenge in training and inference efficiency when applied to large-scale real-world datasets. In this paper, we introduce LightTS, a light deep learning architecture merely based on simple MLP-based structures. The key idea of LightTS is to apply an MLP-based structure on top of two delicate down-sampling strategies, including interval sampling and continuous sampling, inspired by a crucial fact that down-sampling time series often preserves the majority of its information. We conduct extensive experiments on eight widely used benchmark datasets. Compared with the existing state-of-the-art methods, LightTS demonstrates better performance on five of them and comparable performance on the rest. Moreover, LightTS is highly efficient. It uses less than 5% FLOPS compared with previous SOTA methods on the largest benchmark dataset. In addition, LightTS is robust and has a much smaller variance in forecasting accuracy than previous SOTA methods in long sequence forecasting tasks.

Subfields: Environmental Engineering, Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/0ae9ea4846f52f14095e86a2cd384983f074746a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Environmental Engineering

Mingsheng Long

In Management Science and Operations Research

In Signal Processing

Citations in the corpus

TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting (2024, background): “…al., 2023a; Hewage et al., 2020), RNN (Lai et al., 2018; Qin et al., 2017; Salinas et al., 2020), Transformer (Vaswani et al., 2017; Zhou et al., 2021; Wu et al., 2021; Zhou et al., 2022b; Nie et al., 2023) and MLP (Zeng et al., 2023; Zhang et al., 2022; Oreshkin et al., 2019; Challu et al., 2023).”
TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting (2023, methodology): “We categorize SOTA forecasting benchmarks into the following categories: (i) Standard Transformers: FEDformer [38], Autoformer[30] and Informer [37], (ii) Patch Transformers: PatchTST [22] and CrossFormer [3], (iii) MLPs and Non-Transformers: DLinear [34], LightTS [36] and S4 [12], (iv) Self-supervi”
FITS: Modeling Time Series with 10k Parameters (2023, methodology): “Following TimesNet (Wu et al., 2023), we also compare the anomaly detection performance with other models (Zeng et al., 2023; Zhang et al., 2022; Woo et al., 2022; Zhou et al., 2022a).”
Is Mamba Effective for Time Series Forecasting? (2024, unknown): “As a new architecture, Mamba [22] swiftly attracted the attention of a large number of researchers in Natural Language Processing (NLP), Computer Vision (CV), and other Artificial Intelligence communities.”
TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis (2022, methodology): “Technically, MLP-based methods (Oreshkin et al., 2019; Challu et al., 2022; Zeng et al., 2023; Zhang et al., 2022) adopt the MLP along the temporal dimension and encode the temporal dependencies into the fixed parameter of MLP layers.”
Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023, methodology): “LightTS [20] uses lightweight sampling-oriented MLP structures to reduce complexity and computation time while maintaining accuracy.”