Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting

By Bryan Lim, Sercan Ö. Arik, Nicolas Loeff, Tomas Pfister. 2019

2,901 citations recorded by the source corpus.

Multi-horizon forecasting problems often contain a complex mix of inputs -- including static (i.e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed historically -- without any prior information on how they interact with the target. While several deep learning models have been proposed for multi-step prediction, they typically comprise black-box models which do not account for the full range of inputs present in common scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) -- a novel attention-based architecture which combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. To learn temporal relationships at different scales, the TFT utilizes recurrent layers for local processing and interpretable self-attention layers for learning long-term dependencies. The TFT also uses specialized components for the judicious selection of relevant features and a series of gating layers to suppress unnecessary components, enabling high performance in a wide range of regimes. On a variety of real-world datasets, we demonstrate significant performance improvements over existing benchmarks, and showcase three practical interpretability use-cases of TFT.

Subfields: Electrical and Electronic Engineering, Management Science and Operations Research, Model Architectures, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

  1. Ziqing Ma
  2. Bryan Lim
  3. Xifeng Yan
  4. Tomas Pfister
  5. Yu-Xiang Wang
  6. Nate Yoder
  7. Liang Sun
  8. Chun-Liang Li

In Management Science and Operations Research

  1. Bryan Lim
  2. Bernie Wang
  3. Stefan Zohren
  4. Tim Januschowski
  5. Jan Gasthaus
  6. Junchi Yan
  7. Syama Sundar Rangapuram
  8. Laurent Callot

In Model Architectures

  1. Bryan Lim
  2. Bernie Wang
  3. Laurent Callot
  4. Frédérick Gusset
  5. Ziqing Ma
  6. Michael Bohlke-Schneider
  7. David Salinas
  8. Xiao Ma

In Signal Processing

  1. Bernie Wang
  2. Bryan Lim
  3. Tim Januschowski
  4. Jan Gasthaus
  5. Junchi Yan
  6. Stefan Zohren
  7. Syama Sundar Rangapuram
  8. Danielle C. Maddix

In Time-Series & Forecasting

  1. Bryan Lim
  2. Bernie Wang
  3. Stefan Zohren
  4. Tim Januschowski
  5. Jan Gasthaus
  6. Danielle C. Maddix
  7. Syama Sundar Rangapuram
  8. Junchi Yan

Citations in the corpus