Time-Series Representation Learning via Temporal and Contextual Contrasting

By Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee Keong Kwoh, Xiaoli Li, Cuntai Guan. International Joint Conference on Artificial Intelligence, 2021

834 citations recorded by the source corpus.

Learning decent representations from unlabeled time-series data with temporal dynamics is a very challenging task. In this paper, we propose an unsupervised Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC), to learn time-series representation from unlabeled data. First, the raw time-series data are transformed into two different yet correlated views by using weak and strong augmentations. Second, we propose a novel temporal contrasting module to learn robust temporal representations by designing a tough cross-view prediction task. Last, to further learn discriminative representations, we propose a contextual contrasting module built upon the contexts from the temporal contrasting module. It attempts to maximize the similarity among different contexts of the same sample while minimizing similarity among contexts of different samples. Experiments have been carried out on three real-world time-series datasets. The results manifest that training a linear classifier on top of the features learned by our proposed TS-TCC performs comparably with the supervised training. Additionally, our proposed TS-TCC shows high efficiency in few-labeled data and transfer learning scenarios. The code is publicly available at https://github.com/emadeldeen24/TS-TCC.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/47e0dab08c920b589a9ddd11643c694d47ccd1c4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “Time series data are widely used with many important real-world applications [54, 152, 228, 243].”
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2022, methodology): “However, although people have made attempts on Transformer-based models like time series Transformer (TST) (Zerveas et al., 2021) and TS-TCC (Eldele et al., 2021), the potential is still not fully realized yet.”
MOMENT: A Family of Open Time-series Foundation Models (2024, background): “For time series data, prior work has primarily focused on contrastive representation learning (Yue et al., 2022; Eldele et al., 2021; Franceschi et al., 2019).”
Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency (2022, methodology): “Here, the time-based augmentation bank includes jittering, scaling, time-shifts, and neighborhood segments, all well-established in contrastive learning [40, 48, 41].”
TS2Vec: Towards Universal Representation of Time Series (2021, methodology): “TS-TCC (Eldele et al. 2021) encourages consistency of different data augmentations.”