Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors

By Daniel P. Simpson, Håvard Rue, Thiago G. Martins, Andrea Riebler, Sigrunn Holbek Sørbye. 2014

1,120 citations recorded by the source corpus.

In this paper, we introduce a new concept for constructing prior distributions. We exploit the natural nested structure inherent to many model components, which defines the model component to be a flexible extension of a base model. Proper priors are defined to penalise the complexity induced by deviating from the simpler base model and are formulated after the input of a user-defined scaling parameter for that model component, both in the univariate and the multivariate case. These priors are invariant to reparameterisations, have a natural connection to Jeffreys’ priors, are designed to support Occam’s razor and seem to have excellent robustness properties, all which are highly desirable and allow us to use this approach to define default prior distributions. Through examples and theoretical results, we demonstrate the appropriateness of this approach and how it can be applied in various situations.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/db5ed765e709b7feb71bd1c813d1b0a2bb412574

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

An intuitive Bayesian spatial model for disease mapping that accounts for scaling (2016, methodology): “Since we are not in a variable selection setting, we follow the recommendation of Simpson et al.(13) and use the exponential distribution, which supports Riebler et al.”
Scale-Dependent Priors for Variance Parameters in Structured Additive Distributional Regression (2016, methodology): “(ii) As already noted by Simpson et al. (2014), the exponential decay assumption can of course be replaced by alternative priors for the distance if desired.”
The expected demise of the Bayes factor (2015, methodology): “Different perspectives on this issue of constructing reference testing solutions are obviously welcome, from the incorporation of testing into the PC priors and baseline models of (Simpson et al., 2014) to the non-local tests of Johnson and Rossell (2010), and I would most gladly welcome exchanges o”
Bayesian Workflow (2020, background): “More complicated transformations can also serve the purpose of making parameters more interpretable and thus facilitating the use of prior information; Riebler et al. (2016) give an example for a class of spatial correlation models, and Simpson et al. (2017) consider this idea more generally.”
The Prior Can Often Only Be Understood in the Context of the Likelihood (2017, background): “The ground between structural and regularizing priors is occupied by the more recent, and considerably more hazy, idea of weakly informative priors that are explicitly designed to encode information that applies to a general class of problems without taking full advantage of problem-specific knowled”
Estimation of a non-stationary model for annual precipitation in southern Norway using replicates of the spatial field (2014, background): “Prior specifications for hierarchical model parameters are challenging for applied Bayesians, and common priors might have undesired properties (Gelman, 2006; Sørbye and Rue, 2014; Simpson et al., 2014).”
Probabilistic Interpolation of Uncertain Local Activation Times on Human Atrial Manifolds (2020, methodology): “Penalized complexity priors [26] were chosen for l, τ , ω and ρ:”
Projecting the future burden of cancer: Bayesian age–period–cohort analysis with integrated nested Laplace approximations (2017, methodology): “(2015) for details, to these choices and also used PC priors (Simpson et al., 2015).”
Spatial modelling with R-INLA: A review (2018, methodology): “A joint principled prior for r and σ was developed by Fuglstad et al. (2017) using the penalised complexity framework developed by Simpson et al. (2017), and the prior has been successfully applied in practice (Beguin et al., 2017; Wakeﬁeld et al., 2018).”
Maximum likelihood estimation of spatially varying coefficient models for large data with an application to real estate price prediction (2020, methodology): “Since we expect in most of our applications the fields to be not too smooth, we follow the recommendation of Stein (1999) and use exponential covariance functions.”
Validating Bayesian Inference Algorithms with Simulation-Based Calibration (2018, methodology): “We use penalized complexity priors (Simpson et al., 2017; Fuglstad et al., 2017) on the remaining parameters tuned to ensure Pr(ρ < 0.”
Accounting for individual-specific variation in habitat-selection studies: Efficient estimation of mixed-effects models using Bayesian or frequentist computation (2018, background): “These were recently proposed as robust and intuitive alternatives to inverse gamma priors, and were shown to have excellent robustness properties (Simpson et al., 2017).”
Constructing Priors that Penalize the Complexity of Gaussian Random Fields (2015, methodology): “Simpson et al. (2017) present a general approach where conditional on the value of the distance, D , the probability density is uniformly distributed on the set of parameters that specify models that are a distance D from the base model, but the approach is not parameterization invariant and it is…”
Visualization in Bayesian workflow (2017, methodology): “For the posterior predictive checks, we ameliorate the concerns by checking carefully for inﬂuential measurements and proposing that model extensions be weakly informative extensions that are still centered on the previous model (Simpson et al., 2017).”
Bayesian Computing with INLA: A Review (2016, background): “If we are concerned about the behaviour of the distance between the more ﬂexible and the simpler model component, we should deﬁne the prior directly on the distance, as proposed in Simpson et al. (2016b).”
Spatiotemporal wildfire modeling through point processes with moderate and extreme marks (2021, methodology): “We estimate our marked log-Gaussian Cox process in a Bayesian setting using INLA (Illian et al. 2012) with Penalized Complexity (PC) priors for hyperparameters (Simpson et al. 2017).”
INLA goes extreme: Bayesian tail regression for the estimation of high spatio-temporal quantiles (2018, methodology): “Following Simpson et al. (2017), a natural deﬁnition of the “distance” d ( f ξ , f ξ 0 ) of a model f ξ with respect to a reference model f ξ 0 (here assumed to be in the same family for simplicity) may be based on the Kullback-Leibler divergence (KLD) by setting For the GP distribution (1) with 0…”