Cross-validation of bias-corrected climate simulations is misleading
By Douglas Maraun, M. Widmann. Hydrology and Earth System Sciences, 2018
57 citations recorded by the source corpus.
Abstract. We demonstrate both analytically and with a modelling example that cross-validation of free-running bias-corrected climate change simulations against observations is misleading. The underlying reasoning is as follows: a cross-validation can have in principle two outcomes. A negative (in the sense of not rejecting a null hypothesis), if the residual bias in the validation period after bias correction vanishes; and a positive, if the residual bias in the validation period after bias correction is large. It can be shown analytically that the residual bias depends solely on the difference between the simulated and observed change between calibration and validation periods. This change, however, depends mainly on the realizations of internal variability in the observations and climate model. As a consequence, the outcome of a cross-validation is also dominated by internal variability, and does not allow for any conclusion about the sensibility of a bias correction. In particular, a sensible bias correction may be rejected (false positive) and a non-sensible bias correction may be accepted (false negative). We therefore propose to avoid cross-validation when evaluating bias correction of free-running bias-corrected climate change simulations against observations. Instead, one should evaluate non-calibrated temporal, spatial and process-based aspects.
Subfields: Global and Planetary Change, Management Science and Operations Research
View Paper: https://www.semanticscholar.org/paper/bc33ad788b881c654020e04e8b7ad9028ede458e
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Global and Planetary Change
- Jakob Zscheischler
- S. Lange
- Erich Fischer
- Yoann Robin
- Mathieu Vrac
- Philippe Naveau
- Pascal Yiou
- Hans Van De Vyver
In Management Science and Operations Research
- Yoann Robin
- Jakob Zscheischler
- Mathieu Vrac
- Philippe Naveau
- Erich Fischer
- Pascal Yiou
- Mathieu Ribatet
- A. C. Davison
Citations in the corpus
- The effect of univariate bias adjustment on multivariate hazard estimates (2019, background): “Furthermore, cross-validation might not help to diagnose whether a bias-adjustment approach is effective (Switanek et al., 2017; Maraun and Widmann, 2018).”
- Multivariate stochastic bias corrections with optimal transport (2018, methodology): “As suggested by Maraun and Widmann (2018), our cross-validation method does not compare the correction to the observations on the validation period, which can produce false positive or true negative due to internal variability of model or observations, but assesses whether the statistical evolution…”