FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting

By Tian Zhou, Ziqing Ma, Xue Wang, Qingsong Wen, Liang Sun, Tao Yao, Rong Jin. Neural Information Processing Systems, 2022

365 citations recorded by the source corpus.

Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, that there is still great room for improvement in how to preserve historical information in neural networks while avoiding overfitting to noise presented in the history. Addressing this allows better utilization of the capabilities of deep learning models. To this end, we design a \textbf{F}requency \textbf{i}mproved \textbf{L}egendre \textbf{M}emory model, or {\bf FiLM}: it applies Legendre Polynomials projections to approximate historical information, uses Fourier projection to remove noise, and adds a low-rank approximation to speed up computation. Our empirical studies show that the proposed FiLM significantly improves the accuracy of state-of-the-art models in multivariate and univariate long-term forecasting by (\textbf{20.3\%}, \textbf{22.6\%}), respectively. We also demonstrate that the representation module developed in this work can be used as a general plug-in to improve the long-term prediction performance of other deep learning modules. Code is available at https://github.com/tianzhou2011/FiLM/

Subfields: Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/defecf3dc299214f4cb76a093c6eed2297eaa46f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

1. Qiang Xu
2. Ailing Zeng
3. Shiyu Wang
4. Lintao Ma
5. Tomas Pfister
6. Haixu Wu
7. Nate Yoder
8. Chun-Liang Li

In Signal Processing

1. Shiyu Wang
2. Zhijian Xu
3. Lintao Ma
4. Qiang Xu
5. Ailing Zeng
6. Tengge Hu
7. Haixu Wu
8. Xiaoming Shi

Citations in the corpus

• FITS: Modeling Time Series with 10k Parameters (2023, methodology): “Following TimesNet (Wu et al., 2023), we also compare the anomaly detection performance with other models (Zeng et al., 2023; Zhang et al., 2022; Woo et al., 2022; Zhou et al., 2022a).”
• FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective (2023, positive): “Multivariate Time Series Forecasting with Fourier Transform Recently, many MTS forecasting models have integrated the Fourier theory into deep neural networks [22, 23].”
• TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting (2024, background): “FiLM (Zhou et al., 2022a) projects time series into Legendre Polynomials space, where different basis functions correspond to different period components in the original series.”
• Resurrecting Recurrent Neural Networks for Long Sequences (2023, neutral): “, 2022) and time series forecasting (Zhou et al., 2022).”
• TSMixer: An all-MLP Architecture for Time Series Forecasting (2023, methodology): “…− P ) + c , where a, c ∈ R are constants, the linear model still has a solution for perfect prediction: A more general assumption is that the time series can be decomposed into a periodic sequence and a sequence with smooth trend (Holt, 2004; Zhang & Qi, 2005; Wu et al., 2021; Zhou et al., 2022b).”
• Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023, methodology): “FiLM [32] utilizes Fourier analysis to preserve historical information and remove noisy signals.”