MOMENT: A Family of Open Time-series Foundation Models

By Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, Artur Dubrawski. International Conference on Machine Learning, 2024

550 citations recorded by the source corpus.

We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis. Pre-training large models on time series data is challenging due to (1) the absence of a large and cohesive public time series repository, and (2) diverse time series characteristics which make multi-dataset training onerous. Additionally, (3) experimental benchmarks to evaluate these models, especially in scenarios with limited resources, time, and supervision, are still in their nascent stages. To address these challenges, we compile a large and diverse collection of public time series, called the Time series Pile, and systematically tackle time series-specific challenges to unlock large-scale multi-dataset pre-training. Finally, we build on recent work to design a benchmark to evaluate time series foundation models on diverse tasks and datasets in limited supervision settings. Experiments on this benchmark demonstrate the effectiveness of our pre-trained models with minimal data and task-specific fine-tuning. Finally, we present several interesting empirical observations about large pre-trained time series models. Pre-trained models (AutonLab/MOMENT-1-large) and Time Series Pile (AutonLab/Timeseries-PILE) are available on Huggingface.

Subfields: Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/aabfdbd9db5ce9b1d598eae44e0d6250e7f0fc00

Citations in the corpus

• Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (2024, unknown): “Current research efforts in universal forecasting with time series foundation models can be broadly classified into three categories, as summarized in Table 8: (1) encoder-only models , such as Moirai (Woo et al., 2024) and Moment (Goswami et al., 2024), which employ masked reconstruction and have b”
• Cross-Modal Epileptic Signal Harmonization: Frequency Domain Mapping Quantization for Pre-training a Unified Neurophysiological Transformer (2025, unknown): “To benchmark the performance of our proposed EpiNT model, we compared it with five representative pre-trained models designed for neurophysiological signals and general time series analysis: MAE-EEG [16], Brant [21], MOMENT [50], PatchTST [51], and VQ-MTM [19].”
• Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings (2026, unknown): “Given our limited compute availability, all baseline results reported in the results table are drawn from prior published work and we limited ourselves to reported models in the MOMENT paper.”
• TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval (2025, unknown): “We also replace the channel-biased attention ( CbA ) with two alternatives: full attention, similar to a multivariate variant of Moment [8], and causal attention, analogous to decoder-only designs like Timer-XL [9], Both alternatives yield degraded performance.”
• Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions (2025, unknown): “Foundation embedding models specifically targeting time series representation learning have begun to emerge, leveraging reconstruction-based or next-step forecasting objectives (Goswami et al., 2024; Gao et al., 2024; Trirat et al., 2024).”