Sparse partial least squares regression for simultaneous dimension reduction and variable selection

By Hyonho Chun, Sunduz Keles. Journal of The Royal Statistical Society Series B-statistical Methodology, 2010

885 citations recorded by the source corpus.

Summary. Partial least squares regression has been an alternative to ordinary least squares for handling multicollinearity in several areas of scientific research since the 1960s. It has recently gained much attention in the analysis of high dimensional genomic data. We show that known asymptotic consistency of the partial least squares estimator for a univariate response does not hold with the very large p and small n paradigm. We derive a similar result for a multivariate response regression with partial least squares. We then propose a sparse partial least squares formulation which aims simultaneously to achieve good predictive performance and variable selection by producing sparse linear combinations of the original predictors. We provide an efficient implementation of sparse partial least squares regression and compare it with well‐known variable selection and dimension reduction approaches via simulation experiments. We illustrate the practical utility of sparse partial least squares regression in a joint analysis of gene expression and genomewide binding data.

Subfields: Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/eeca494ed4f29143b8c342fe4b939e6a77936d42

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

A review of variable selection methods in Partial Least Squares Regression (2012, background): “Many authors, for example [1–5], have addressed this problem and there are numerous suggested approaches for dealing with the so-called large p small n problem [1], that is, many variables and few samples.”
Local shrinkage rules, Lévy processes and regularized regression (2010, methodology): “We compare the Bayesian model with the three basic techniques (PLS, RR and PCR), along with a new technique called sparse PLS (Chun and Keleş, 2010) aimed at simultaneous dimension reduction and variable selection.”
Metabolomics and Multi-Omics Integration: A Survey of Computational Methods and Resources (2020, background): “Another variation is Sparse Partial Least Squares Discriminant Analysis (SPLS-DA) [155], which ensures that the number of analytes contributing to the model is relatively small compared to the total number of analytes in the input.”
Sparse PLS discriminant analysis: biologically relevant feature selection and graphical displays for multiclass problems (2011, methodology): “Recently, [30] extended the SPLS from [27] for multiclass classification problems and demonstrated that both SPLSDA and SPLS with an incorporated generalized framework (SGPLS) improved classification accuracy compared to classical PLS [31-33].”
Robust linear regression for high‐dimensional data: An overview (2020, background): “Chun and Keleş (2010) proposed a slightly modified objective function which imposes sparsity on a surrogate vector, and not directly on the vector of the linear combination, which allows to get sufficiently enough sparsity. It turns out that the resulting sparse PLS loadings vector—compare with Equa”
Applied Predictive Modeling (2013, methodology): “2011; Witten and Tibshirani 2011), PLS (Chun and Keleş 2010), and PCA (Jolliffe et al. 2003; Zou et al. 2004). A significant advancement for this model was Efron et al. (2004). Their model, least angle regression (LARS), is a broad framework that encompasses the lasso and similar models. The LARS mo”
Partial least squares prediction in high-dimensional regression (2019, methodology): “We chose to assume d known so that the results might reflect the core behavior of PLS while keeping an important link with the work of Chun and Keleş [6].”