Smoothing spline Gaussian regression: more scalable computation via efficient approximation

By Young-Ju Kim, Chong Gu. 2004

321 citations recorded by the source corpus.

Summary. Smoothing splines via the penalized least squares method provide versatile and effective nonparametric models for regression with Gaussian responses. The computation of smoothing splines is generally of the order O(n3), n being the sample size, which severely limits its practical applicability. We study more scalable computation of smoothing spline regression via certain low dimensional approximations that are asymptotically as efficient. A simple algorithm is presented and the Bayes model that is associated with the approximations is derived, with the latter guiding the porting of Bayesian confidence intervals. The practical choice of the dimension of the approximating space is determined through simulation studies, and empirical comparisons of the approximations with the exact solution are presented. Also evaluated is a simple modification of the generalized cross‐validation method for smoothing parameter selection, which to a large extent fixes the occasional undersmoothing problem that is suffered by generalized cross‐validation.

Subfields: Analytical Chemistry, Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Habitat-based spatial models of cetacean density in the eastern Pacific Ocean (2012, methodology): “Because GCV is known to overfit on occasion (Kim & Gu 2004), we tested 2 values of the parameter gamma that mgcv uses to compute GCV. Larger values for gamma penalize model complexity more than smaller values, so we tested the default, gamma = 1.0, and an alternative, gamma = 1.4.”
Generalized Additive Models (2006, unknown): “4 degrees of freedom in the GCV score, which forces models to be a little smoother than they might otherwise be, and is an ad hoc way of avoiding overfitting (Kim and Gu, 2004).”
Classification and valuation of urban green spaces—A hedonic house price valuation (2013, background): “Kim and Gu (2004) argue that k should be proportional to n, where n is the number of observations.”
Econometric modelling of the regional knowledge production function in Europe (2015, background): “…the degree of nonlinearity of the estimated smooth function, and when 22 Because the GCV may present a tendency to over fit, we have increased the amount of smoothing by correcting the GCV score by a factor ¼ 1:4, which can correct the over-fitting without compromising model fit (Kim and Gu, 2004).”
Optimal smoothing in nonparametric mixed-effect models (2005, methodology): “To circumvent the problem, a simple modification proved to be very effective in the empirical studies of Kim and Gu [9].”
Fast Calibrated Additive Quantile Regression (2017, methodology): “The work of Lin et al. (2013) is not an alternative to what we propose here, because their focus is variable selection, rather than smoothing.”
Inferring trackline detection probabilities, g(0), for cetaceans from apparent densities in different survey conditions (2015, background): “…LongitudeÞ þ sðYearÞ þ offsetðESAÞ: ð3Þ To prevent model over-fitting, the maximum degrees of freedom for the univariate terms (year and Beaufort) is limited (mgcv parameter k = 4) and the overall penalty for model complexity is inflated (mgcv parameter gamma = 1.4) (Kim and Gu 2004, Wood 2006).”
Low‐Rank Scale‐Invariant Tensor Product Smooths for Generalized Additive Mixed Models (2006, methodology): “…to each replicate: (a) a tensor product smooth of xi, zi, and vi; (b) an SS-ANOVA-type model with all seven terms represented using the tensor product smooths given in Section 2; and (c) an SS-ANOVA model (of similar structure to that used in [b]) estimated using the method of Kim and Gu (2004).”
Growth and mortality rates of the fan mussel Pinna nobilis in Lake Vouliagmeni (Korinthiakos Gulf, Greece): a generalized additive modelling approach (2007, methodology): “GCV and UBRE are known to have some tendency for occasional overfitting, and it has been suggested that using c *1.4 can largely correct this without compromising the model fit ( Kim and Gu 2004; Wood 2006).”
Estimation of a semiparametric recursive bivariate probit model in the presence of endogeneity (2011, methodology): “…degrees of freedom or number of effective parameters of the penalized model. γ is a tuning parameter which can be increased from its usual value of 1 to obtain smoother models; Kim & Gu (2004) suggested that using γ ≈ 1.4 can correct the tendency to overfitting of prediction error criteria.”
40 Years of benthic community change on the Caribbean reefs of Curaçao and Bonaire: the rise of slimy cyanobacterial mats (2017, methodology): “4 (Kim and Gu 2004) gave the most reliable and robust outcomes.”