Generalized Sparse Additive Models

By Asad Haris, Noah R. Simon, Ali Shojaie. 2019

20 citations recorded by the source corpus.

We present a unified framework for estimation and analysis of generalized additive models in high dimensions. The framework defines a large class of penalized regression estimators, encompassing many existing methods. An efficient computational algorithm for this class is presented that easily scales to thousands of observations and features. We prove minimax optimal convergence bounds for this class under a weak compatibility condition. In addition, we characterize the rate of convergence when this compatibility condition is not met. Finally, we also show that the optimal penalty parameters for structure and sparsity penalties in our framework are linked, allowing cross-validation to be conducted over only a single tuning parameter. We complement our theoretical results with empirical studies comparing some existing methods within this framework.

Subfields: Computational Mechanics, Statistics and Probability, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistics and Probability

In Representation & Self-Supervised Learning

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “…functions are estimated by splines, many works apply convex regularizers (e.g., `1) to control both smoothness and sparsity (Lin et al., 2006; Ravikumar et al., 2009; Meier et al., 2009; Yin and Yu, 2017; Petersen et al., 2016; Lou et al., 2016; Sadhanala et al., 2019; Haris et al., 2019).”
SurvNAM: The machine learning survival model explanation (2021, background): “It should be noted that the sparse GAM has been studied by several authors [84, 85, 83].”