Fisher Lecture: Dimension Reduction in Regression

By Ronald Christensen. 2007

358 citations recorded by the source corpus.

Beginning with a discussion of R. A. Fisher's early written remarks that relate to dimension reduction, this article revisits principal components as a reductive method in regression, develops several model-based extensions and ends with descriptions of general approaches to model-based and model-free dimension reduction in regression. It is argued that the role for principal components and related methodology may be broader than previously seen and that the common practice of conditioning on observed values of the predictors may unnecessarily limit the choice of regression methodology.

Subfields: Control and Systems Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

Deep Learning in Finance (2016, background): “See Wold (1956), Diaconis and Shahshahani (1984), Ripley (1996), Cook (2007), and Hastie et al. (2009) for further discussion.”
Sure independence screening for ultrahigh dimensional feature space (2006, neutral): “| 〈AT v, AT ′v′〉 | θS,S′ ‖v‖‖v′‖ holds for all disjoint subsets T and T ′ of cardinalities |T | S and |T ′| S′, v∈R|T | and v′ ∈R|T |. The following theorem is obtained by the sure screening property of SIS in theorem 1 along with theorem 1.1 in Candes and Tao (2007), where ε∼N .”
ON THE IDENTIFIABILITY OF ADDITIVE INDEX MODELS (2011, methodology): “1) reduces to the single index model (Duan and Li (1991); Härdle, Hall and Ichimura (1993); Ichimura (1993)).”
Estimation for a partial-linear single-index model (2009, background): “We refer to Cook (1998, 2007) for more details.”
Variable Selection in Predictive Regressions (2013, methodology): “As Cook (2007) points out, Fisher (1924) recognizes the need for dimension reduction in regression analysis but cautions that predictors might be spuriously chosen if reference is made to the dependent variable.”
ldr: An R Software Package for Likelihood-Based Su?cient Dimension Reduction (2014, methodology): “Principal ﬁtted components methodology was proposed by Cook (2007) and estimates the central subspace by modeling the inverse mean function E ( X | Y = y ) while assuming that the conditional variance function ∆ is independent of Y .”
Nonlinear sufficient dimension reduction for functional data (2017, background): “The precursors of this theory include Bach and Jordan (2003), Cook (2007), Wu (2008) and Yeh, Huang and Lee (2009), which introduced a variety of practical nonlinear sufficient dimension reduction methods without articulating a unifying framework.”
Deep learning for short-term traffic flow prediction (2016, background): “suit functions for a speciﬁc types of input functions F(x). Deep learning allows for efﬁcient modeling of nonlinear functional representations. For example, principal components analysis Wold (1956); Cook (2007) for dimensionality reduction can be extended to a case when components are nonlinear fun”
Parametric or nonparametric? A parametricness index for model selection (2011, background): “” Cook [19] discussed Fisher’s insights in details.”
Principal fitted components for dimension reduction in regression (2008, methodology): “This leads to the final partially maximized log likelihood (Cook, 2007, Section 7.2) Ld(∆) = − np 2 log(2π)− n 2 log |∆| (4) − n 2 tr(∆−1Σ̂res)− n 2 p∑ i=d+1 λi(∆ −1Σ̂fit), where λi(A) indicates the ith eigenvalue of the matrix A.”
ENVELOPE MODELS FOR PARSIMONIOUS AND EFFICIENT MULTIVARIATE LINEAR REGRESSION (2010, methodology): “Instead, interest lies in the central subspace SY |X = Σ X|Y B (Cook, 2007), where still B = span(β).”
Inner envelopes: Efficient estimation in multivariate linear regression (2012, background): “By Appendix A.6 of Cook (2007), for any semi-orthogonal matrix G1 ∈ Rr×d , and its completion G0, log |GT1̂resG1| + log |GT0̂resG0| log | T1̂res 1| + log | T0̂res 0| for all G.”
A general theory for nonlinear sufficient dimension reduction: Formulation and estimation (2013, methodology): “If a matrix γ satisfies span(γ)⊆ SY |X , we say that γ is unbiased [Cook (1998b)].”
Text as Data (2017, positive): “Instead, Taddy (2013b) uses the inverse regression ideas of Cook (2007) in deriving sufficient projections from the fitted models.”
Achieving near perfect classification for functional data (2012, neutral): “(See Cook (2007) for discussion.)”