Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis

By Jing Yang. Environmental Modelling & Software, 2011

333 citations recorded by the source corpus.

Sensitivity analysis plays an important role in model development, calibration, uncertainty analysis, scenario analysis, and, hence, decision making. With the availability of different sensitivity analysis techniques, selecting an appropriate technique, monitoring the convergence and estimating the uncertainty of the sensitivity indices are very crucial for environmental modelling, especially for distributed models due to their high non-linearity, non-monotonicity, highly correlated parameters, and intensive computational requirements. It would be useful to identify whether some techniques outperform others with respect to computational requirements, reliability, and other criteria. This paper proposes two methods to monitor the convergence and estimate the uncertainty of sensitivity analysis techniques. One is based on the central limit theorem and the other on the bootstrap technique. These two methods are implemented to assess five different sensitivity analysis techniques applied to an environmental model. These techniques are: the Sobol' method, the Morris method, Linear Regression (LR), Regionalized Sensitivity Analysis (RSA), and non-parametric smoothing. The results show that: (i) the Sobol' method is very robust in quantifying sensitivities and ranking parameters despite a large number of model evaluations; (ii) the Morris method is efficient to rank out unimportant parameters at a medium cost; (iii) the non-parametric smoothing is reliable and robust in quantifying the main effects and low-order interactions while requiring a small number of model evaluations; finally (iv) the other two techniques, that is, LR and RSA, should be used with care.

Subfields: Civil and Structural Engineering, Environmental Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3b9f56e0ba6f58885ed527d7d241c7f1e6b78774

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Civil and Structural Engineering

In Environmental Engineering

In Statistics and Probability

Citations in the corpus

A performance comparison of sensitivity analysis methods for building energy models (2015, methodology): “…that used or supported the Sobol method as the reference method (Saltelli and Bolado 1998; Ridolfi 2013; Massmann and Holzmann 2012; Wainwright et al. 2013; Confalonieri et al. 2010; Campolongo et al. 2007; Yang 2011), the Sobol method was considered the reference method for further investigations.”
Variance-based sensitivity analysis of a forest growth model (2012, background): “Thus, local SA methods are inadequate for analysing complex biophysical process models which may have many parameters, and may be high-dimensional and/or non-linear (Saltelli and Annoni, 2010; Yang, 2011).”
Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications (2015, methodology): “Variance-based methods are also widely used for parameter SA in hydrological models (Table 3) as they can provide most accurate and robust sensitivity indices for complex nonlinear models (Tang et al., 2007b; Yang, 2011; Herman et al., 2013b, 2013c; Zhan et al., 2013).”
Characterising performance of environmental models (2013, methodology): “For more information on sensitivity analysis, see Saltelli et al. (2000), Norton (2008), Frey and Patil (2002), Saltelli and Annoni (2010), Shahsavani and Grimvall (2011), Makler-Pick et al. (2011), Yang (2011), Nossent et al. (2011), Ratto et al. (2012) and Ravalico et al. (2010).”
Parameter Analysis and Estimates for the MODIS Evapotranspiration Algorithm and Multiscale Verification (2019, methodology): “However, despite its low computational cost, the OAT method has been shown to produce highly biased results when it is applied to high‐dimension and nonlinear systems (Yang, 2011), which is a common case in environmental modeling.”
Sensitivity analysis of environmental models: A systematic review with practical workflow (2014, background): “A discussion of the quality of bootstrapping-based con ﬁ dence limits of some widely used sensitivity indices can be found in Yang (2011).”
Sensitivity analysis of infectious disease models: methods, advances and their application (2013, methodology): “GSA techniques have been broadly applied in systems biology [11,21], environmental modelling [22,23] and infectious disease modelling.”
Global Sensitivity Analysis of environmental models: Convergence and validation (2016, methodology): “1 bottom left panel), Tang et al. (2007) use a base sample size n of 8192 (for a case study with 18 input factors), while Yang (2011) uses n equal to 3000 (for a case study with 5 input factors).”
An overview of methods to evaluate uncertainty of deterministic models in decision support (2015, methodology): “This can be achieved by using various approaches ranging from simple one-factor-at-a-time methods to more comprehensive approaches, usually based on MC methods (e.g. Saltelli et al., 2005; Cariboni et al., 2007; Yang, 2011).”
Variance-based global sensitivity analysis for multiple scenarios and models with implementation using sparse grid collocation (2015, methodology): “…and takes into account the interactions between different parameters (van Griensven et al., 2006; Herman et al., 2013; Mishra et al., 2009; Nossent et al., 2011; Pan et al., 2011; Saltelli, 2000; Saltelli et al., 2010; Saltelli and Sobol, 1995; Shi et al., 2014; Song et al., 2015; Yang, 2011).”
A Matlab toolbox for Global Sensitivity Analysis (2015, neutral): “Many GSA methods have been proposed in the literature and their application and comparison in the environmental modelling domain has steadily increased in recent years (e.g. Tang et al. (2007); Pappenberger et al. (2008); Yang (2011)).”
Sobol' sensitivity analysis of a complex environmental model (2011, background): “Despite its high computational demands, this powerful SA technique has recently become more popular in environmental modeling (Pappenberger et al., 2008; Tang et al., 2007; van Werkhoven et al., 2008; Yang, 2011), because of its ability to incorporate parameter interactions and the relatively straig”