ETSformer: Exponential Smoothing Transformers for Time-series Forecasting

By Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, Steven C. H. Hoi. arXiv.org, 2022

321 citations recorded by the source corpus.

Transformers have been actively studied for time-series forecasting in recent years. While often showing promising results in various scenarios, traditional Transformers are not designed to fully exploit the characteristics of time-series data and thus suffer some fundamental limitations, e.g., they generally lack of decomposition capability and interpretability, and are neither effective nor efficient for long-term forecasting. In this paper, we propose ETSFormer, a novel time-series Transformer architecture, which exploits the principle of exponential smoothing in improving Transformers for time-series forecasting. In particular, inspired by the classical exponential smoothing methods in time-series forecasting, we propose the novel exponential smoothing attention (ESA) and frequency attention (FA) to replace the self-attention mechanism in vanilla Transformers, thus improving both accuracy and efficiency. Based on these, we redesign the Transformer architecture with modular decomposition blocks such that it can learn to decompose the time-series data into interpretable time-series components such as level, growth and seasonality. Extensive experiments on various time-series benchmarks validate the efficacy and advantages of the proposed method. Code is available at https://github.com/salesforce/ETSformer.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/68d14c4203c8798ad94ddfc7bf1616b92ae25299

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mingsheng Long
2. Jianmin Wang
3. Haixu Wu
4. Yong Liu
5. Qiang Xu
6. Zhijian Xu
7. Ailing Zeng
8. Kate Saenko

In Management Science and Operations Research

1. Qiang Xu
2. Ailing Zeng
3. Haixu Wu
4. Mingsheng Long
5. Jianmin Wang
6. Yong Liu
7. Zhijian Xu
8. Liang Sun

In Signal Processing

1. Qiang Xu
2. Ailing Zeng
3. Haixu Wu
4. Zhijian Xu
5. Mingsheng Long
6. Jianmin Wang
7. Yong Liu
8. Donghao Luo

Citations in the corpus

• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (2023, unknown): “Further research should explore optimal reprogramming representations, enrich LLMs with explicit time series knowledge through continued pre-training, build towards multi-modal models with joint reasoning across modalities, and apply the reprogramming framework to imbue LLMs with additional new…”
• Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting (2022, background): “Full results (under all prediction lengths and promotion on ETSformer [34], FEDformer [38]) can be found in Appendix.”
• FITS: Modeling Time Series with 10k Parameters (2023, result): “Following TimesNet (Wu et al., 2023), we also compare the anomaly detection performance with other models (Zeng et al., 2023; Zhang et al., 2022; Woo et al., 2022; Zhou et al., 2022a).”