A decoder-only foundation model for time-series forecasting

By Abhimanyu Das, Weihao Kong, Rajat Sen, Yichen Zhou. International Conference on Machine Learning, 2023

785 citations recorded by the source corpus.

Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public datasets comes close to the accuracy of state-of-the-art supervised forecasting models for each individual dataset. Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus, and can work well across different forecasting history lengths, prediction lengths and temporal granularities.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f45f85fa1beaa795c24c4ff86f1f2deece72252f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Silvio Savarese
2. Doyen Sahoo
3. Chenghao Liu
4. Caiming Xiong
5. Artur Dubrawski
6. Mononito Goswami
7. Gerald Woo
8. Akshat Kumar

In Signal Processing

1. Doyen Sahoo
2. Artur Dubrawski
3. Gerald Woo
4. Silvio Savarese
5. Chenghao Liu
6. Caiming Xiong
7. Akshat Kumar

Citations in the corpus

• Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities (2024, methodology): “As shown by Fig.17, TimesFM [184] employs decoder-only architecture to train a time-series prediction model.”
• Unified Training of Universal Time Series Forecasting Transformers (2024, methodology): “TimesFM (Das et al., 2023b) is a patch-based decoder-only foundation model for time series forecasting, introducing a larger output patch size for faster decoding.”
• Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (2024, neutral): “While time series pre-training datasets are no longer a major bottleneck, most existing works (Das et al., 2024; Woo et al., 2024; Ansari et al., 2024) have not extensively discussed their in-model data processing pipelines or mixing strategies.”
• MOMENT: A Family of Open Time-series Foundation Models (2024, background): “On 44 time series from the UCR anomaly detection archive, MOMENT consistently outper-formed both TimesNet and GPT4TS, as well as 2 state-of-the-art deep learning models tailored for anomaly detection, in both zero-shot and linear probing configurations.”
• Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings (2026, unknown): “Early foundational attempts predominantly addressed univariate series (Das et al., 2024; Ansari et al., 2024), though recent advancements have successfully extended to multivariate settings with sophisticated cross-channel modeling techniques (Woo et al., 2024b; Liu et al., 2024).”
• TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval (2025, unknown): “Formally, the channel-level negative set is defined as: and similarly in the reverse direction:”
• Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions (2025, unknown): “Early foundational attempts predominantly addressed univariate series (Das et al., 2024; Ansari et al., 2024), though recent advancements have successfully extended to multivariate settings with sophisticated cross-channel modeling techniques (Woo et al., 2024; Liu et al., 2024).”