Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

By Mingxing Xu, Wenrui Dai, Chunmiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, H. Xiong. arXiv.org, 2020

520 citations recorded by the source corpus.

Traffic forecasting has emerged as a core component of intelligent transportation systems. However, timely accurate traffic forecasting, especially long-term forecasting, still remains an open challenge due to the highly nonlinear and dynamic spatial-temporal dependencies of traffic flows. In this paper, we propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) that leverages dynamical directed spatial dependencies and long-range temporal dependencies to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies with self-attention mechanism to capture realtime traffic conditions as well as the directionality of traffic flows. Furthermore, different spatial dependency patterns can be jointly modeled with multi-heads attention mechanism to consider diverse relationships related to different factors (e.g. similarity, connectivity and covariance). On the other hand, the temporal transformer is utilized to model long-range bidirectional temporal dependencies across multiple time steps. Finally, they are composed as a block to jointly model the spatial-temporal dependencies for accurate traffic prediction. Compared to existing works, the proposed model enables fast and scalable training over a long range spatial-temporal dependencies. Experiment results demonstrate that the proposed model achieves competitive results compared with the state-of-the-arts, especially forecasting long-term traffic flows on real-world PeMS-Bay and PeMSD7(M) datasets.

Subfields: Building and Construction, Signal Processing

View Paper: https://www.semanticscholar.org/paper/24b50f529a69aca17f2729dc3a9f8c7ada1a7d02

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Building and Construction

In Signal Processing

Citations in the corpus

Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting (2020, methodology): “The work most similar to ours is Xu et al. (2020), which also focused on jointly modeling spatial and temporal dependencies by making full use of Transformer to solve traffic prediction task.”
Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting (2023, background): “Among them, Spatio-Temporal Graph Neural Networks (STGNNs) [19, 38] and Transformer-based models [12, 14, 36, 42] become very popular for their outstanding performance.”
Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting (2020, methodology): “A very recent work [24] uses transformers for”
UniVLA: Learning to Act Anywhere with Task-centric Latent Actions (2025, unknown): “We implement the encoder as a spatial-temporal trans-former [84] with casual temporal masks, following Villegas et al. [77].”
Genie: Generative Interactive Environments (2024, methodology): “We thus adopt a memory efficient ST-transformer architecture (inspired by Xu et al. (2020), see Figure 4) across all model components, balancing model capacity with computational constraints.”
Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting (2020, background): “Temporal self-attention module (Xu et al. 2020; Wang et al. 2020) of transformers can also extract dynamic spatial-temporal correlations predetermined spatial graph may not reﬂect.”
AgiBot World Colosseo: A Large-Scale Manipulation Platform for Scalable and Intelligent Embodied Systems (2025, unknown): “In Stage 2, these latent actions act as pseudo-labels for the latent planner, facilitating embodiment-agnostic long-horizon planning and leveraging the generalizability of the pre-trained VLM. Finally, in Stage 3, we introduce the action expert and jointly train it with the latent planner to…”
Graph Neural Network for Traffic Forecasting: A Survey (2021, unknown): “Problem Relevant Studies Road Traffic Flow [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62],”
Spatio-Temporal Meta-Graph Learning for Traffic Forecasting (2022, unknown): “Third, motivated by (Vaswani et al. 2017), a series of traffic trans-formers (Zheng et al. 2020; Xu et al. 2020) and time series transformers (Li et al. 2019; Zhou et al. 2021; Xu et al. 2021) were proposed to do the long sequence time series modeling.”
PDFormer: Propagation Delay-aware Dynamic Long-range Transformer for Traffic Flow Prediction (2023, methodology): “(4) Selfattention-based Models: We choose STTN (Xu et al. 2020), GMAN (Zheng et al.”
Transformers in Time Series: A Survey (2022, background): “…combination of GNN and Transformers/attentions could bring not only signiﬁcant performance improvements like in trafﬁc forecasting [Cai et al. , 2020; Xu et al. , 2020] and multi-modal forecasting [Li et al. , 2021], but also better understanding of the spatio-temporal dynamics and latent…”