Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model

By Xingjian Shi, Zhihan Gao, Leonard Lausen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, Wang-chun Woo. Neural Information Processing Systems, 2017

1,027 citations recorded by the source corpus.

With the goal of making high-resolution forecasts of regional rainfall, precipitation nowcasting has become an important and fundamental technology underlying various public services ranging from rainstorm warnings to flight safety. Recently, the Convolutional LSTM (ConvLSTM) model has been shown to outperform traditional optical flow based methods for precipitation nowcasting, suggesting that deep learning models have a huge potential for solving the problem. However, the convolutional recurrence structure in ConvLSTM-based models is location-invariant while natural motion and transformation (e.g., rotation) are location-variant in general. Furthermore, since deep-learning-based precipitation nowcasting is a newly emerging area, clear evaluation protocols have not yet been established. To address these problems, we propose both a new model and a benchmark for precipitation nowcasting. Specifically, we go beyond ConvLSTM and propose the Trajectory GRU (TrajGRU) model that can actively learn the location-variant structure for recurrent connections. Besides, we provide a benchmark that includes a real-world large-scale dataset from the Hong Kong Observatory, a new training loss, and a comprehensive evaluation protocol to facilitate future research and gauge the state of the art.

Subfields: Atmospheric Science, Global and Planetary Change

View Paper: https://www.semanticscholar.org/paper/051bc84485f0d9be02003e27e0ca67e9a06f2e4d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Global and Planetary Change

Citations in the corpus

WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting (2020, positive): “Some attempts at using machine learning (ML) techniques for this task have appeared in recent years (Shi et al., 2015, 2017; Agrawal et al., 2019).”
Graph Neural Controlled Differential Equations for Traffic Forecasting (2021, background): “The spatio-temporal graph data frequently happens in real-world applications, ranging from trafﬁc to climate forecasting (Zaytar and El Amrani 2016; Shi et al. 2015, 2017; Liu et al. 2016; Racah et al. 2016; Kurth et al. 2018; Cheng et al. 2018a,b; Hossain et al. 2015; Ren et al. 2021; Tekin et al.…”
Self-Attention ConvLSTM for Spatiotemporal Prediction (2020, methodology): “MovingMNIST is a commonly used dataset contains a variety of sequences generated by the method mentioned in (Srivastava et al. 2015), depicting two potentially overlapping digits moving with constant velocity and bouncing off the image edges.”
MetNet: A Neural Weather Model for Precipitation Forecasting (2020, background): “…neural networks for precipitation forecasting focuses on lead times between 60 and 90 minutes (Agrawal et al.; Ayzel et al., 2019a; Lebedev et al., 2019; Xingjian et al., 2015; Shi et al., 2017) with optical ﬂow at times outperforming neural networks (Ayzel et al., 2019a; Lebedev et al., 2019).”
Deep Learning for Spatio-Temporal Data Mining: A Survey (2019, neutral): “Others [36], [10], [46], [195], [26], [193], [168] [124], [93] [133], [145], [202], [21], [183], [146], [79], [43], [185], [186], [132] [122], [63], [71]”
SmaAt-UNet: Precipitation Nowcasting using a Small Attention-UNet Architecture (2020, methodology): “5 mm / h was chosen in line with the works by [6, 31] and it di ﬀ erentiates between rain and no rain.”
GraphCast: Learning skillful medium-range global weather forecasting (2022, background): “, 2022) and precipitation nowcasting, both with and without NWP information (Espeholt et al., 2022; Ravuri et al., 2021; Shi et al., 2017; Sønderby et al., 2020).”
Skilful precipitation nowcasting using deep generative models of radar (2021, methodology): “These include the Convolutional Gated Recurrent Units (ConvGRU) and Trajectory GRU (TrajGRU) [11] (the latter model incorporates an optical flow module in a ConvGRU-like layer to model convective dynamics), as well as UNets with classification and regression outputs, respectively [17] and [18].”
Memory in Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity From Spatiotemporal Dynamics (2018, neutral): “…the state-of-the-art results on the standard Moving MNIST Model SSIM MSE MAE FC-LSTM [26] 0.690 118.3 209.4 ConvLSTM [24] 0.707 103.3 182.9 TrajGRU [25] 0.713 106.9 190.1 CDNA [9] 0.721 97.4 175.3 DFN [5] 0.726 89.0 172.8 FRNN [20] 0.813 69.7 150.3 VPN baseline [14] 0.870 64.1 131.0 PredRNN [32]…”
Spatiotemporal Multi-Graph Convolution Network for Ride-Hailing Demand Forecasting (2019, methodology): “With convolutional neural network and recurrent neural network, state-of-the-art results are achieved in (Shi et al. 2015; Yu et al. 2017; Shi et al. 2017; Zhang, Zheng, and Qi 2017; Zhang et al. 2018a; Ma et al. 2017; Yao et al. 2018b; 2018a).”
Forward Modeling for Partial Observation Strategy Games - A StarCraft Defogger (2018, background): “[9] propose a LSTM with convolutional instead of linear interactions and demonstrate good performance on predicting percipitation based on radar echo images; [10] use a similar approach to forecast passenger demand for on-demand ride services; [11] estimate future traffic in Bejing based on image re”
A Review on Deep Learning Techniques for Video Prediction (2020, background): “[10], prediction of pedestrian trajectories in traffic [11], anomaly detection [12], precipitation nowcasting [13], [14], and autonomous driving [15].”
Earthformer: Exploring Space-Time Transformers for Earth System Forecasting (2022, methodology): “Previous works relied on the combination of Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN) [36, 37, 43, 13, 45].”
PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning (2021, methodology): “Model Architecture Recurrent unit Training (Encoder + Decoder) ConvLSTM [1] Seq-to-Seq with stacked layers [23] Convolutional LSTM Standard [23] + Standard [23] TrajGRU [25] Seq-to-Seq with reversed forecasting layers Trajectory GRU Standard + Standard CDNA [26] Action-conditioned ConvLSTM network C”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, background): “A significant amount of attention has been paid to using DL for precipitation forecasting (e.g., Hernández et al., 2016; Shi et al., 2017; P. Zhang et al., 2017).”
GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs (2018, background): “This can be viewed as an extension of Trajectory GRU (TrajGRU) (Shi et al., 2017) on irregular or graph-structured data.”