A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model
By Yanjun Gan, Qingyun Duan, Wei Gong, Charles H. Tong, Yunwei Sun, Wei Chu, Aizhong Ye, Chiyuan Miao, Zhenhua Di. Environmental Modelling & Software, 2014
342 citations recorded by the source corpus.
Sensitivity analysis (SA) is a commonly used approach for identifying important parameters that dominate model behaviors. We use a newly developed software package, a Problem Solving environment for Uncertainty Analysis and Design Exploration (PSUADE), to evaluate the effectiveness and efficiency of ten widely used SA methods, including seven qualitative and three quantitative ones. All SA methods are tested using a variety of sampling techniques to screen out the most sensitive (i.e., important) parameters from the insensitive ones. The Sacramento Soil Moisture Accounting (SAC-SMA) model, which has thirteen tunable parameters, is used for illustration. The South Branch Potomac River basin near Springfield, West Virginia in the U.S. is chosen as the study area. The key findings from this study are: (1) For qualitative SA methods, Correlation Analysis (CA), Regression Analysis (RA), and Gaussian Process (GP) screening methods are shown to be not effective in this example. Morris One-At-a-Time (MOAT) screening is the most efficient, needing only 280 samples to identify the most important parameters, but it is the least robust method. Multivariate Adaptive Regression Splines (MARS), Delta Test (DT) and Sum-Of-Trees (SOT) screening methods need about 400-600 samples for the same purpose. Monte Carlo (MC), Orthogonal Array (OA) and Orthogonal Array based Latin Hypercube (OALH) are appropriate sampling techniques for them; (2) For quantitative SA methods, at least 2777 samples are needed for Fourier Amplitude Sensitivity Test (FAST) to identity parameter main effect. McKay method needs about 360 samples to evaluate the main effect, more than 1000 samples to assess the two-way interaction effect. OALH and LP"@t (LPTAU) sampling techniques are more appropriate for McKay method. For the Sobol' method, the minimum samples needed are 1050 to compute the first-order and total sensitivity indices correctly. These comparisons show that qualitative SA methods are more efficient but less accurate and robust than quantitative ones.
Subfields: Civil and Structural Engineering, Management, Monitoring, Policy and Law, Safety, Risk, Reliability and Quality
View Paper: https://www.semanticscholar.org/paper/09ccf9765e1bbfe7891f52b3ed64d656b9931653
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Civil and Structural Engineering
In Safety, Risk, Reliability and Quality
Citations in the corpus
- Facilitating Parameter Estimation and Sensitivity Analysis of Agent-Based Models: A Cookbook Using NetLogo and 'R' (2014, methodology): “All source codes, i.e., the NetLogo model implementation and the R/RNetLogo scripts, are available in the Supplementary Material. sensitivity analysis methods (e.g., Saltelli et al. 2000, 2004, 2008; Helton et al. 2006; Kleijnen 1995, 2008; Cournède et al. 2013; Gan et al. 2014).”
- A Matlab toolbox for Global Sensitivity Analysis (2015, methodology): “…by the Joint Research Centre (JRC, 2014), the Sensitivity Analysis package for the R environment (Pujol et al., 2014), the GUI-HDMR Matlab package (Ziehn and Tomlin, 2009), the C þþ based PSUADE software (Gan et al., 2014), and the Python Sensitivity Analysis Library SALib (Herman, 2014).”
- Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications (2015, neutral): “Some sampling design methods have been verified as effective (Razavi et al., 2012), such as central composite design (Montgomery, 2008), full factorial design (Gutmann, 2001), Latin Hypercube sampling (Gan et al., 2014), quasi-random sampling (Elsawwaf et al., 2010; Zhan et al., 2013).”
- The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support (2020, neutral): “If the inclusion of an input or set of inputs significantly improves the quality of fit, they are deemed important (Gan et al., 2014).”