Neural networks for post-processing ensemble weather forecasts

By Stephan Rasp, Sebastian Lerch. Monthly Weather Review, 2018

440 citations recorded by the source corpus.

Ensemble weather predictions require statistical postprocessing of systematic errors to obtain reliable and accurate probabilistic forecasts. Traditionally, this is accomplished with distributional regression models in which the parameters of a predictive distribution are estimated from a training period. We propose a flexible alternative based on neural networks that can incorporate nonlinear relationships between arbitrary predictor variables and forecast distribution parameters that are automatically learned in a data-driven way rather than requiring prespecified link functions. In a case study of 2-m temperature forecasts at surface stations in Germany, the neural network approach significantly outperforms benchmark postprocessing methods while being computationally more affordable. Key components to this improvement are the use of auxiliary predictor variables and station-specific information with the help of embeddings. Furthermore, the trained neural network can be used to gain insight into the importance of meteorological variables, thereby challenging the notion of neural networks as uninterpretable black boxes. Our approach can easily be extended to other statistical postprocessing and forecasting problems. We anticipate that recent advances in deep learning combined with the ever-increasing amounts of model and observation data will transform the postprocessing of numerical weather forecasts in the coming decade.

Subfields: Atmospheric Science, Electrical and Electronic Engineering, Environmental Engineering

View Paper: https://www.semanticscholar.org/paper/c62c44af1e5c8e552a377c82db228741d0c62149

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Electrical and Electronic Engineering

In Environmental Engineering

Citations in the corpus

WeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models (2023, background): “This is a direct consequence of these models minimizing a deterministic mean error (such as MSE).”
Estimation Methods for Nonhomogeneous Regression Models: Minimum Continuous Ranked Probability Score versus Maximum Likelihood (2018, methodology): “…03:55 PM UTC purposes to obtain stable regression coefficients, and a sufficiently large amount of test data is not used for regression (e.g., as in Hamill et al. 2004; Wilks and Hamill 2007; Hagedorn et al. 2008; Wilks 2009; Messner et al. 2017; Gebetsberger et al. 2017; Rasp and Lerch 2018).”
Neural general circulation models for weather and climate (2023, methodology): “Our stochastic loss is based on the CRPS 28,48,49 .”
Distributional regression forests for probabilistic precipitation forecasting in complex terrain (2018, background): “Thus, distributional forests are an appealing alternative to (boosted) GAMLSS in weather forecasting tasks as the main concern is typically not so much interpretability but forecasting skill and (semi-)automatic application on a larger domain (see also the discussion in Rasp and Lerch 2018).”
WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting (2020, background): “Gneiting et al., 2005; Rasp and Lerch, 2018).”
Improving Data‐Driven Global Weather Prediction Using Deep Convolutional Neural Networks on a Cubed Sphere (2020, background): “s receiving increasing attention and yielding promising results. Machine learning has been used in concert with output from numerical weather prediction (NWP) models in attempts to improve forecasts: [31] used neural networks (NNs) to successfully improve post-processing of forecasts from general ci”
Can deep learning beat numerical weather prediction? (2021, methodology): “Later, other simple semantic network techniques were used for post-processing and prediction optimization of NWP output [62,63], and as surrogate models for different parameterization schemes in climate models [64,65].”
Can Machines Learn to Predict Weather? Using Deep Learning to Predict Gridded 500‐hPa Geopotential Height From Historical Weather Data (2019, methodology): “For example, Rasp and Lerch (2018) used neural networks (NNs) to successfully improve post-processing of GCM forecasts to surface stations, while Rodrigues, Oliveira, Cunha, and Netto (2018) demonstrated the ability of deep NNs to down-scale GCM out-43 put to higher horizontal resolution.”
Skillful Twelve Hour Precipitation Forecasts using Large Context Neural Networks (2021, methodology): “There have also been efforts to postprocess NWP predictions using neural networks by incorporating them into the set of input features [20]; this is similar to our MetNet-2 Postprocess variant.”