Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding

By Sana Tonekaboni, Danny Eytan, Anna Goldenberg. International Conference on Learning Representations, 2021

402 citations recorded by the source corpus.

Time series are often complex and rich in information but sparsely labeled and therefore challenging to model. In this paper, we propose a self-supervised framework for learning generalizable representations for non-stationary time series. Our approach, called Temporal Neighborhood Coding (TNC), takes advantage of the local smoothness of a signal's generative process to define neighborhoods in time with stationary properties. Using a debiased contrastive objective, our framework learns time series representations by ensuring that in the encoding space, the distribution of signals from within a neighborhood is distinguishable from the distribution of non-neighboring signals. Our motivation stems from the medical field, where the ability to model the dynamic nature of time series data is especially valuable for identifying, tracking, and predicting the underlying patients' latent states in settings where labeling data is practically impossible. We compare our method to recently developed unsupervised representation learning approaches and demonstrate superior performance on clustering and classification tasks for multiple datasets.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e19ca3c11fd45dfc9bc6e43ddf9b03b6c798e66d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

TS2Vec: Towards Universal Representation of Time Series (2021, methodology): “…instance-wise contrasting only at the instance level; TS-TCC (Eldele et al. 2021) applies instance-wise contrasting only at the timestamp level; TNC (Tonekaboni, Eytan, and Goldenberg 2021) where Ω is the set of timestamps within the overlap of the two subseries, and 1 is the indicator function.”
TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting (2023, methodology): “We categorize SOTA forecasting benchmarks into the following categories: (i) Standard Transformers: FEDformer [38], Autoformer[30] and Informer [37], (ii) Patch Transformers: PatchTST [22] and CrossFormer [3], (iii) MLPs and Non-Transformers: DLinear [34], LightTS [36] and S4 [12], (iv) Self-supervi”
Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency (2022, background): “The TS2Vec, TS-TCC, SimCLR, TNC, and CPC are designed for representation learning on a single dataset rather than for transfer learning, so we apply them to fit our settings and make the results comparable.”
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2022, methodology): “We compare our self-supervised model with BTSF (Yang & Hong, 2022), TS2Vec (Yue et al., 2022), TNC (Tonekaboni et al., 2021), and TS-TCC (Eldele et al., 2021) which are among the state-of-the-art contrastive learning representation methods for time series 2 .”
Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings (2026, unknown): “Analogous approaches have been adapted for time series, predominantly using contrastive self-supervised tasks (Yue et al., 2022; Fraikin et al., 2024; Franceschi et al., 2019; Tonekaboni et al., 2021).”
Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions (2025, unknown): “Analogous approaches have been adapted for time series, predominantly using contrastive self-supervised tasks (Yue et al., 2022; Fraikin et al., 2024; Franceschi et al., 2019; Tonekaboni et al., 2021).”