Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

By Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan and 1 more. International Conference on Learning Representations, 2023

1,070 citations recorded by the source corpus.

Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. Unlike natural language process (NLP) and computer vision (CV), where a single large model can tackle multiple tasks, models for time series forecasting are often specialized, necessitating distinct designs for different tasks and applications. While pre-trained foundation models have made impressive strides in NLP and CV, their development in time series domains has been constrained by data sparsity. Recent studies have revealed that large language models (LLMs) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the challenge remains in effectively aligning the modalities of time series data and natural language to leverage these capabilities. In this work, we present Time-LLM, a reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact. We begin by reprogramming the input time series with text prototypes before feeding it into the frozen LLM to align the two modalities. To augment the LLM's ability to reason with time series data, we propose Prompt-as-Prefix (PaP), which enriches the input context and directs the transformation of reprogrammed input patches. The transformed time series patches from the LLM are finally projected to obtain the forecasts. Our comprehensive evaluations demonstrate that Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models. Moreover, Time-LLM excels in both few-shot and zero-shot learning scenarios.

View Paper: https://www.semanticscholar.org/paper/16f01c1b3ddd0b2abd5ddfe4fdb3f74767607277

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (2023, background): “Many studies have been proposed to harness the reasoning ability of LLMs to handle complex tasks through prompting (Wei et al., 2022; Wang et al., 2022; Yao et al., 2023; Besta et al., 2023; Pan et al., 2023; Zheng et al., 2023d; Jin et al., 2024a;b).”
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook (2024, unknown): “Significant advancements have been made in M-LLMs with temporal analysis capabilities, including models such as Time-LLM [90], LLM4TS [91], TEMPO [92], and PromptCast [93], among others [94,95].”
Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities (2024, background): “Conversely, soft prompts ( P soft ) introduce trainable embeddings that are optimized during training to influence the model’s prediction subtly [189].”
Unified Training of Universal Time Series Forecasting Transformers (2024, methodology): “Zhou et al. (2023); Jin et al. (2023) introduce modules and fine-tuning methods to adapt LLMs for time series tasks including forecasting.”
Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings (2026, unknown): “For instance, (Jin et al., 2024), (Pan et al., 2024), and (Sun et al., 2024) an LLM is finetuned for time series data or the time series are transformed into token sequences consumable by an LLM.”
TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval (2025, unknown): “Recently, several multimodal encoders have been proposed to integrate time series and text [35, 36, 37, 38, 39, 40], which aim to leverage the generalization capabilities of large language models by reprogramming time series into token-like representations or textual prototypes.”