Stable and Efficient Multiple Smoothing Parameter Estimation for Generalized Additive Models

By Simon N. Wood. 2004

1,979 citations recorded by the source corpus.

Representation of generalized additive models (GAM's) using penalized regression splines allows GAM's to be employed in a straightforward manner using penalized regression methods. Not only is inference facilitated by this approach, but it is also possible to integrate model selection in the form of smoothing parameter selection into model fitting in a computationally efficient manner using well founded criteria such as generalized cross-validation. The current fitting and smoothing parameter selection methods for such models are usually effective, but do not provide the level of numerical stability to which users of linear regression packages, for example, are accustomed. In particular the existing methods cannot deal adequately with numerical rank deficiency of the GAM fitting problem, and it is not straightforward to produce methods that can do so, given that the degree of rank deficiency can be smoothing parameter dependent. In addition, models with the potential flexibility of GAM's can also present practical fitting difficulties as a result of indeterminacy in the model likelihood: Data with many zeros fitted by a model with a log link are a good example. In this article it is proposed that GAM's with a ridge penalty provide a practical solution in such circumstances, and a multiple smoothing parameter selection method suitable for use in the presence of such a penalty is developed. The method is based on the pivoted QR decomposition and the singular value decomposition, so that with or without a ridge penalty it has good error propagation properties and is capable of detecting and coping elegantly with numerical rank deficiency. The method also allows mixtures of user specified and estimated smoothing parameters and the setting of lower bounds on smoothing parameters. In terms of computational efficiency, the method compares well with existing methods. A simulation study compares the method to existing methods, including treating GAM's as mixed models.

Subfields: Management Science and Operations Research, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Shape constrained additive models (2015, methodology): “We develop stable computational schemes for estimating the model coefficients and smoothing parameters, able to deal with the ill-conditioning that can affect even unconstrained GAM fits (Wood 2004, 2008), while retaining computational efficiency.”
brms: An R Package for Bayesian Multilevel Models Using Stan (2017, methodology): “Another flexible model class are generalized additive mixed models (Hastie and Tibshirani 1990; Wood 2004; Zuur 2014), which can be fitted with brms and rstanarm.”
Zelig: Everyone's Statistical Software (2006, background): “Advanced users may wish to refer to help(gam), [60], [59], and other documentation accompanying the mgcv package.”
Bayesian structured additive distributional regression with an application to regional income inequality in Germany (2015, background): “…the mean of a response variable to covariates [see, e.g., Fahrmeir, Kneib and Lang (2004), Fahrmeir et al. (2013), Ruppert, Wand and Carroll (2003), Wood (2004, 2008)], distributional regression enables the consideration of basically any response distribution and allows to specify regression…”
Estimation and model selection in generalized additive partial linear models for correlated data with diverging number of covariates (2014, methodology): “To solve the “curse of dimensionality” problem in computing, [30] suggested a penalized regression splines approach to utilize the practical benefits of smoothing spline methods and the computational advantages of local scoring backfitting [2].”
A comparison of methods for the fitting of generalized additive models (2008, methodology): “2003) and implementations have been made readily available (Wood 2004).”
Low‐Rank Scale‐Invariant Tensor Product Smooths for Generalized Additive Mixed Models (2006, methodology): “Given its model matrix and penalties, the coefficients and smoothing parameters of a tensor product smooth can be estimated using the multiple penalized regression methods given in Wood (2004), or incorporated into a generalized additive model and estimated in the same way.”
Econometric modelling of the regional knowledge production function in Europe (2015, methodology): “The estimation is based on the maximization of a penalized likelihood by penalized iteratively reweighted least squares (P-IRLS) (Wood, 2004).”
Coverage Properties of Confidence Intervals for Generalized Additive Model Components (2012, methodology): “Fast stable computational methods for multiple smoothing parameter estimation are provided in Wood (2004, 2008).”
Spatial fisheries ecology: Recent progress and future prospects (2008, methodology): “The error terms ei are generally assumed to be independent and identically distributed with zero mean and common variance, but in very recent applications it is possible to generalize the model to allow for (spatially) correlated errors (Wood, 2004b).”
Nonlife Ratemaking and Risk Management with Bayesian Additive Models for Location, Scale and Shape (2013, methodology): “Inference can be realized by cross validation as in Wood (2004), by mixed model representations as in Ruppert et al. (2003), Fahrmeir et al. (2004) and Wood (2008) or by Markov chain Monte Carlo (MCMC) simulations as in Brezger and Lang (2006), Julion and Lambert (2007) and Lang et al. (2013).”
Seize the night: European Blackbirds (Turdus merula) extend their foraging activity under artificial illumination (2015, methodology): “To handle this nonlinearity, we applied a general additive mixed model (GAMM) in the mgcv package (Wood 2004) of the R software system (R Development Core Team 2013).”
Mixed Effects Models and Extensions in Ecology with R (2009, methodology): “R code to calculate the smoothing curves can be found in Wood (2006), and modified code to present the graphs in an xyplot from the lattice package is on the book website”