A significance test for forward stepwise model selection

By Joshua R. Loftus, Jonathan E. Taylor. 2014

67 citations recorded by the source corpus.

We apply the methods developed by Lockhart et al. (2013) and Taylor et al. (2013) on significance tests for penalized regression to forward stepwise model selection. A general framework for selection procedures described by quadratic inequalities includes a variant of forward stepwise with grouped variables, allowing us to handle categorical variables and factor models. We provide an algorithm to compute a new statistic with an exact null distribution conditional on the outcome of the model selection procedure. This new statistic, which we denote $T\chi$, has a truncated $\chi$ distribution under the global null. We apply this test in forward stepwise iteratively on the residual after each step. The resulting method has the computational strengths of stepwise selection and addresses the problem of invalid test statistics due to model selection. We illustrate the flexibility of this method by applying it to several specialized applications of forward stepwise including a hierarchical interactions model and a recently described additive model that adaptively chooses between linear and nonlinear effects for each variable.

View Paper: https://www.semanticscholar.org/paper/86ec4c243cd0f9a9458ca77f2e5a8b16d294d9db

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Exact post-selection inference for the generalized lasso path (2018, background): “A number of extensions to diﬀerent problem settings are given in Lee and Taylor [23], Reid et al. [29], Loftus and Taylor [28], Choi et al. [8].”
Uniform asymptotic inference and the bootstrap after model selection (2015, methodology): “…an example of a broader framework that we might call selective pivotal inference, applicable in many other settings, as in, e.g., Taylor et al. (2016), Lee et al. (2016), Lee & Taylor (2014), Loftus & Taylor (2014), Reid et al. (2017), Choi et al. (2014), Fithian et al. (2014), Hyun et al. (2016).”
Selective Sequential Model Selection (2015, background): “Other related works include Loftus and Taylor (2014), who describe p-values for path algorithms that add groups of variables (instead of individual variables) at each step, and Choi et al. (2014), who describe p-values for steps of principle components analysis (each step marking the estimation of…”
Sequential selection procedures and false discovery rate control (2013, methodology): “The flexibility of our approach should be a major asset, as the proliferation of methods for pathwise hypothesis testing suggests an interest in the topic (Lockhart et al., 2014; Loftus and Taylor, 2014; Fithian et al., 2014; G’Sell et al., 2013; Lee et al., 2013; Lee and Taylor, 2014; Taylor et al.”
Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach (2015, unknown): “With the formulations above, Neyman’s C(α)-statistic takes the form: C(α) = ‖S(α)‖22, S(α) = Ω̂−1/2(α, η̂) √ nM̂(α, η̂), where M̂(α, η̂) = En[ψ(wi, α, η̂)] as before, Ω(α, η0) = Var( √ nM̂(α, η0)), and Ω̂(α, η̂) and η̂ are suitable estimators based on structured assumptions.”
Optimal Inference After Model Selection (2014, methodology): “Characterizations of the partitions in forward stepwise selection and marginal screening can be found in Loftus and Taylor (2014) and Lee and Taylor (2014), respectively.”
Exact Post-Selection Inference for Sequential Regression Procedures (2014, methodology): “, by Wasserman & Roeder (2009), Meinshausen & Buhlmann (2010), Minnier et al. (2011) is based on samplesplitting or resampling methods. Another class of approaches, e.g., by Zhang & Zhang (2014), Buhlmann (2013), van de Geer et al. (2014), Javanmard & Montanari (2013a,b) is based on “debiasing” a re”
Splitting strategies for post-selection inference (2021, result): “…variable-selection algorithms, such as the lasso or stepwise procedures with fixed tuning parameters, the conditioning event {S = s} can be studied analytically (often, it can be written as a union of affine sets; see e.g. Lee and Taylor (2014); Loftus and Taylor (2014); and Lee et al. (2016)).”