Controlling the false discovery rate via knockoffs

By Rina Foygel Barber, E.J. Candes. 2014

860 citations recorded by the source corpus.

In many fields of science, we observe a response variable together with a large number of potential explanatory variables, and would like to be able to discover which variables are truly associated with the response. At the same time, we need to know that the false discovery rate (FDR) - the expected fraction of false discoveries among all discoveries - is not too high, in order to assure the scientist that most of the discoveries are indeed true and replicable. This paper introduces the knockoff filter, a new variable selection procedure controlling the FDR in the statistical linear model whenever there are at least as many observations as variables. This method achieves exact FDR control in finite sample settings no matter the design or covariates, the number of variables in the model, or the amplitudes of the unknown regression coefficients, and does not require any knowledge of the noise level. As the name suggests, the method operates by manufacturing knockoff variables that are cheap - their construction does not require any new data - and are designed to mimic the correlation structure found within the existing variables, in a way that allows for accurate FDR control, beyond what is possible with permutation-based methods. The method of knockoffs is very general and flexible, and can work with a broad class of test statistics. We test the method in combination with statistics from the Lasso for sparse regression, and obtain empirical results showing that the resulting method has far more power than existing selection rules when the proportion of null variables is high.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Forward stability and model path selection (2021, neutral): “Along the same lines, our goal is not necessarily to identify the subset of signals as is the primary objective with tools like knockoffs (Barber and Candes 2015; Candes et al. 2018; Barber et al. 2018; Janson et al. 2016), pseudovariables (Wu et al. 2007; Hu et al. 2018), or stability selection…”
Selective Sequential Model Selection (2015, background): “Barber and Candès (2014) present a framework for controlling FDRfull.”
High-dimensional econometrics and regularized GMM (2018, background): “A radically different procedure, which also allows for FDR control but which we did not consider in this chapter, is the knockoff filter of [5].”
The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions (2019, methodology): “That is, an interaction effect θxixj (1)While the knockoff filter introduced in Barber & Candès [2] is a promising way to control the false discovery rate, such a method has not been evaluated theoretically or empirically for interaction models.”
Covariate powered cross‐weighted multiple testing (2017, methodology): “Its construction is based on a variant of the optimal stopping theorem developed by Barber and Candès (2015), which provides the analyst with considerable flexibility in learning these boundaries from the data, while masking information that could lead to overfitting.”
Splitting strategies for post-selection inference (2021, methodology): “Regarding selection, we considered two recently proposed algorithms: the fixed-X knockoff algorithm of Barber and Candès (2015) and the stability selector paired with the lasso of Meinshausen and Bühlmann (2010), as implemented in the R packages knockoff (Barber et al., 2020) and stabs (Hofner…”
Robust inference with knockoffs (2018, methodology): “That said, one solution is provided by the recent model-X knockoffs approach of Candès et al. [4], which is a new read on the earlier knockoff filter of Barber and Candès [1], see also [2].”
Optimal Inference After Model Selection (2014, background): “Recent work in this vein can be found in Berk et al. (2013) and Foygel Barber and Candes (2014).”
High-dimensional simultaneous inference with the bootstrap (2016, background): “For the case of a linear model with homoscedastic and Gaussian errors, more recent and powerful techniques have been proposed (Bühlmann 2013; Zhang and Zhang 2014; van de Geer et al. 2014; Javanmard and Montanari 2014; Meinshausen 2015; Foygel Barber and Candès 2015) and some of these extend to gene”
Multiple testing with the structure‐adaptive Benjamini–Hochberg algorithm (2016, methodology): “(2016), Barber and Candès (2015) and Li and Barber (2017), mentioned earlier in Section 2, propose algorithms for choosing an adaptive cut-off point k and labelling P1, : : : , Pk as signals and Pk+1, : : : , Pn as nulls. Alternately, the selective SeqStep method of Barber and Candès (2015) selects ”