TS2Vec: Towards Universal Representation of Time Series

By Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yu Tong, Bixiong Xu. AAAI Conference on Artificial Intelligence, 2021

1,033 citations recorded by the source corpus.

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.

Subfields: Artificial Intelligence, Economics and Econometrics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f79a319413b5c014e4a98bcd223e6f65e3f7901b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Signal Processing

Citations in the corpus

A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2022, methodology): “We compare our self-supervised model with BTSF (Yang & Hong, 2022), TS2Vec (Yue et al., 2022), TNC (Tonekaboni et al., 2021), and TS-TCC (Eldele et al., 2021) which are among the state-of-the-art contrastive learning representation methods for time series 2 .”
Deep Isolation Forest for Anomaly Detection (2022, methodology): “2) Graph Data and Time Series: As reported in Section VI-A3, traditional isolation-based methods are performed upon vectorised representations learned by unsupervised representation learning method InfoGraph [42] and TS2Vec [43].”
CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting (2022, methodology): “The encoders used in all approaches except Triplet and TST is the causal TCN encoder as proposed in TS2Vec (Yue et al., 2021).”
Deep Learning for Time Series Anomaly Detection: A Survey (2022, neutral): “This combination of neural networks and Bayesian filters allows NSIBF to accurately detect anomalies in noisy CPS sensor data.”
Unified Training of Universal Time Series Forecasting Transformers (2024, methodology): “Denoising autoencoders (Zerveas et al., 2021) and contrastive learning (Yue et al., 2022; Woo et al., 2022) have been shown to be effective pretext tasks for time series forecasting, but have largely been applied to the existing paradigm of pre-training and fine-tuning on the same dataset, without…”
Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency (2022, methodology): “, [47, 48]) focus exclusively on modeling the time domain and ignore the frequency domain altogether.”
TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting (2023, methodology): “Wecategorize SOTA forecasting benchmarks into the following categories: (i) Standard Transformers: FEDformer [36], Autoformer[28] and Informer [35], (ii) Patch Transformers: PatchTST [20] and CrossFormer [2], (iii) MLPs and Non-Transformers: DLinear [32], LightTS [34] and S4 [10], (iv) Self-supervis”
Foundation Models for Time Series Analysis: A Tutorial and Survey (2024, neutral): “CNN-based architectures, in particular, have garnered significant attention in self-supervised learning for general time series representation, with a notable emphasis on the usage of ResNet [27, 119] and dilated convolution layers [71, 115] as foundational backbones.”
Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (2024, neutral): “Self-supervised learning has been extensively developed for time series (Zhang et al., 2024), employing masked reconstruction (Zerveas et al., 2021; Nie et al., 2023) or contrastive learning (Zhang et al., 2022; Wang et al., 2023b; Yue et al., 2022).”
MOMENT: A Family of Open Time-series Foundation Models (2024, methodology): “As is common in prior work (Yue et al., 2022; Franceschi et al., 2019), the quality of representations is measured using the accuracy of a Support Vector Machine trained on them (App.”
Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings (2026, unknown): “Semantic Representation Learning Models : T-Rep , TS2Vec , T-Loss , TS-TCC etc.”
TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval (2025, unknown): “Recent work has explored retrieval systems for time series data, primarily within a unimodal setting [44, 13, 45, 15].”
Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions (2025, unknown): “Analogous approaches have been adapted for time series, predominantly using contrastive self-supervised tasks (Yue et al., 2022; Fraikin et al., 2024; Franceschi et al., 2019; Tonekaboni et al., 2021).”