CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting

By Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, Steven C. H. Hoi. International Conference on Learning Representations, 2022

483 citations recorded by the source corpus.

Deep learning has been actively studied for time series forecasting, and the mainstream paradigm is based on the end-to-end training of neural network architectures, ranging from classical LSTM/RNNs to more recent TCNs and Transformers. Motivated by the recent success of representation learning in computer vision and natural language processing, we argue that a more promising paradigm for time series forecasting, is to first learn disentangled feature representations, followed by a simple regression fine-tuning step -- we justify such a paradigm from a causal perspective. Following this principle, we propose a new time series representation learning framework for time series forecasting named CoST, which applies contrastive learning methods to learn disentangled seasonal-trend representations. CoST comprises both time domain and frequency domain contrastive losses to learn discriminative trend and seasonal representations, respectively. Extensive experiments on real-world datasets show that CoST consistently outperforms the state-of-the-art methods by a considerable margin, achieving a 21.3% improvement in MSE on multivariate benchmarks. It is also robust to various choices of backbone encoders, as well as downstream regressors. Code is available at https://github.com/salesforce/CoST.

Subfields: Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/c090ff3dec01e06f46735b7b9ab133a5db8c73c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Signal Processing

Citations in the corpus

Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency (2022, background): “In previous work, CoST processes sequential signals through the frequency domain, but the augmentations are still implemented in time space [49].”
Unified Training of Universal Time Series Forecasting Transformers (2024, methodology): “Denoising autoencoders (Zerveas et al., 2021) and contrastive learning (Yue et al., 2022; Woo et al., 2022) have been shown to be effective pretext tasks for time series forecasting, but have largely been applied to the existing paradigm of pre-training and fine-tuning on the same dataset, without…”
Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023, methodology): “CoST [31] uses DFT to map the intermediate features to frequency domain to enables interactions in representation.”
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective (2023, unknown): “…forecasting, we compare FourierGNN with other GNN-based MTS models (including StemGNN [10], AGCRN [2], GraphWaveNet [17], MTGNN [13], and TAMP-S2GCNets [11]) and a representation learning model (CoST [36]) on COVID-19 dataset under different prediction lengths, and the results are shown in Table 2.”
ETSformer: Exponential Smoothing Transformers for Time-series Forecasting (2022, background): “Automatically extracting seasonal patterns has been proved to be critical for the success of forecasting [5, 6, 36].”