Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance

By Cort J. Willmott, Kenji Matsuura. 2005

5,960 citations recorded by the source corpus.

Subfields: Atmospheric Science, Global and Planetary Change

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Global and Planetary Change

Citations in the corpus

Root mean square error (RMSE) or mean absolute error (MAE) (2014, methodology): “Willmott and Matsuura(2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and thus the MAE would be a better metric for that purpose. While some concerns over using RMSE raised by Willmott and Matsuura(2005) an”
A comprehensive survey on recent metaheuristics for feature selection (2022, background): “Precision, Recall, Area Under the Curve, Mean Absolute Error, Kappa statistic [107], Correlation Coefficient [108], Root Mean Square Error, Relative Absolute Error, and Root Relative Squared Error [109] are some examples of other metrics.”
Problems in RMSE-based wave model validations (2013, background): “In some circumstances, anyway, their use can lead to conflicting and inconsistent results in trying to reproduce physical phenomena such as atmosphere dynamics or ocean wave generation and propagation (e.g., Willmott and Matsuura, 2005).”
Technical Note: Comparing the effectiveness of recent algorithms to fill and smooth incomplete and noisy time series (2011, background): “More importantly, RMSE values are not representative of mean or typical errors only: they range between MAE andn1/2 ×MAE, increase non-monotonically with MAE and vary with the square root of n (Willmott and Matsuura, 2005).”
Prediction of rice crop yield using MODIS EVI−LAI data in the Mekong Delta, Vietnam (2013, positive): “It is similar to RMSE but less sensitive to large errors and is thus a more natural and unambiguous measure of average error (Willmott and Matsuura 2005).”
On the Sensitivity of Field Reconstruction and Prediction Using Empirical Orthogonal Functions Derived from Gappy Data (2013, methodology): “The MAE is the 179 arithmetic average of the absolute error (Wilks 2006) and is of practical use for inter180 comparisons given that it presents the magnitude of average model-performance error in 181 the same units as the field (Willmott and Matsuura 2005).”
Maximum entropy-Gumbel-Hougaard copula method for simulation of monthly streamflow in Xiangxi river, China (2015, neutral): “The RMSE can be expressed as (Willmott and Matsuura 2005): RMSE ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPN k¼1 xestk xobsk 2 N vuuut ð20Þ where xestk stands for the estimated value from the fitted univariate probability distribution; xobsk…”
Performance Metrics (Error Measures) in Machine Learning Regression, Forecasting and Prognostics: Properties and Typology (2018, methodology): “Suggested in this paper generic formula for primary performance metrics is more comprehensive than used by Willmott & Matsuura (2005), as their definition can be applied only to metrics with mean-averaging type of error aggregation.”
Evaluation of TanDEM-X DEMs on selected Brazilian sites: Comparison with SRTM, ASTER GDEM and ALOS AW3D30 (2017, background): “being analyzed and co-located values from an independent source of higher accuracy (Willmott and Matsuura, 2005; Wechsler, 2007; Hebeler and Purves, 2009; Reuter et al., 2009a; Baade and Schmullius, 2016) although metrics derived from a set of points that are not evenly distributed in the study area”
An Extra Tree Regression Model for Discharge Coefficient Prediction: Novel, Practical Applications in the Hydraulic Sector and Future Research Directions (2021, methodology): “(ii) Root mean square error (RMSE) [43] is a common measure for comparing prediction error for different models, and the lower the RMSE value, the higher the prediction ability of the model in terms of its absolute deviation, and it is given by [42]”
Prediction and Sensitivity Analysis of Bubble Dissolution Time in 3D Selective Laser Sintering Using Ensemble Decision Trees (2019, methodology): “We can calculate these RMSE and MAE and R2 by following equations [61,62]:”