Shape constrained additive models

By Natalya Pya, Simon N. Wood. Statistics and computing, 2015

326 citations recorded by the source corpus.

A framework is presented for generalized additive modelling under shape constraints on the component functions of the linear predictor of the GAM. We represent shape constrained model components by mildly non-linear extensions of P-splines. Models can contain multiple shape constrained and unconstrained terms as well as shape constrained multi-dimensional smooths. The constraints considered are on the sign of the first or/and the second derivatives of the smooth terms. A key advantage of the approach is that it facilitates efficient estimation of smoothing parameters as an integral part of model estimation, via GCV or AIC, and numerically robust algorithms for this are presented. We also derive simulation free approximate Bayesian confidence intervals for the smooth components, which are shown to achieve close to nominal coverage probabilities. Applications are presented using real data examples including the risk of disease in relation to proximity to municipal incinerators and the association between air pollution and health.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3c8fbee63f0ff2d46b8879cc8738acbe1add5a83

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

Generalized additive and index models with shape constraints (2014, methodology): “The SCMLE method takes roughly 1–2 s with a sample size of n = 1000 and 4–8 additive components, which is unsurprisingly slower than Tree or MARS, but it is typically faster than other shape-constrained methods such as SCAM and MCR. Note that MCR is particularly slow compared with the other methods ”
Faithful Variable Screening for High-Dimensional Convex Regression (2014, methodology): “Additive shape-constrained estimation has also been studied; Pya and Wood (2014) propose a penalized B-spline estimator while Chen and Samworth (2014) show the consistency of the MLE.”
Twenty years of P-splines (2015, background): “Pya and Wood (2015) took a different approach.”
Nonparametric Shape-Restricted Regression (2017, background): “Shape-restricted additive and single-index regression models offer flexible, yet interpretable, statistical procedures for handling multidimensional covariates, and have been extensively used in econometrics, epidemiology and other fields (see [35, 82, 113] and the references therein).”
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals (2018, background): “see e.g. (Barlow et al., 1972; Groeneboom and Jongbloed, 2014; Gupta et al., 2016; Canini et al., 2016; Luss and Rosset, 2017; You et al., 2017; Bonakdarpour et al., 2018) and/or convex/concave (e.g. Pya and Wood, 2015; Chen and Samworth, 2016; Gupta et al., 2018). Shape constraints generally entail”
Learning Global Additive Explanations for Neural Nets Using Model Distillation (2018, background): “Feature shapes also appear in other work that learn models from the original data (i.e. without distillation) with feature shapes that fulﬁll monotonicity [18] or concavity/convexity [32] constraints.”