Breaking the curse of dimensionality in regression

By Yinchu Zhu, Jelena Bradic. 2017

19 citations recorded by the source corpus.

Author(s): Zhu, Yinchu; Bradic, Jelena | Abstract: Models with many signals, high-dimensional models, often impose structures on the signal strengths. The common assumption is that only a few signals are strong and most of the signals are zero or close (collectively) to zero. However, such a requirement might not be valid in many real-life applications. In this article, we are interested in conducting large-scale inference in models that might have signals of mixed strengths. The key challenge is that the signals that are not under testing might be collectively non-negligible (although individually small) and cannot be accurately learned. This article develops a new class of tests that arise from a moment matching formulation. A virtue of these moment-matching statistics is their ability to borrow strength across features, adapt to the sparsity size and exert adjustment for testing growing number of hypothesis. GRoup-level Inference of Parameter, GRIP, test harvests effective sparsity structures with hypothesis formulation for an efficient multiple testing procedure. Simulated data showcase that GRIPs error control is far better than the alternative methods. We develop a minimax theory, demonstrating optimality of GRIP for a broad range of models, including those where the model is a mixture of a sparse and high-dimensional dense signals.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, background): “Zhu and Bradic (2017) show that it is possible to attain √ n-consistency for the coefficients of a partially linear model when the regression function is dense.”
Learning L2 Continuous Regression Functionals via Regularized Riesz Representers (2018, background): “…and Zhang (2014), Belloni Chernozhukov, and Wang (2014), Belloni, Chernozhukov, and Kato (2015), Javanmard and Montanari (2014a,b; 2015), van de Geer et al. (2014), Neykov et al. (2015), Ren et al. (2015), Jankova and van de Geer (2015, 2016a,b), Bradic and Kolar (2017), and Zhu and Bradic (2018).”
Double/de-biased machine learning using regularized Riesz representers (2018, methodology): “…conditions used in Javanmard and Montanari (2015) in the case of inference on a single coefﬁcient in Gaussian sparse linear regression models, and those in Zhu and Bradic (2017a) in the case of point testing general linear hypotheses on regression coefﬁcients in linear Gaussian regression models.”
Super-Resolution Musculoskeletal MRI Using Deep Learning (2018, background): “53,54 DeepResolve could also be generalized to in-plane resolution enhancements in addition to through-plane resolution enhancements.”
A modern maximum-likelihood theory for high-dimensional logistic regression (2018, background): “See [113, 116] for more recent attempts to relax the strong sparsity assumptions in the ultra high-dimensional linear model in the context of significance testing.”