The Discrete Dantzig Selector: Estimating Sparse Linear Models via Mixed Integer Linear Optimization

By Rahul Mazumder, Peter Radchenko. 2015

42 citations recorded by the source corpus.

We propose a novel high-dimensional linear regression estimator: the <italic>Discrete Dantzig Selector</italic>, which minimizes the number of nonzero regression coefficients subject to a budget on the maximal absolute correlation between the features and residuals. Motivated by the significant advances in integer optimization over the past 10–15 years, we present a mixed integer linear optimization (<monospace>MILO</monospace>) approach to obtain <italic>certifiably optimal</italic> global solutions to this nonconvex optimization problem. The current state of algorithmics in integer optimization makes our proposal substantially more computationally attractive than the least squares subset selection framework based on integer <italic>quadratic</italic> optimization, recently proposed by Bertsimas <italic>et al.</italic> and the continuous nonconvex quadratic optimization framework of Liu <italic>et al.</italic>. We propose new discrete first-order methods, which when paired with the state-of-the-art <monospace>MILO</monospace> solvers, lead to good solutions for the <italic>Discrete Dantzig Selector</italic> problem for a given computational budget. We illustrate that our integrated approach provides globally optimal solutions in significantly shorter computation times, when compared to off-the-shelf <monospace>MILO</monospace> solvers. We demonstrate both theoretically and empirically that in a wide range of regimes the statistical properties of the <italic>Discrete Dantzig Selector</italic> are superior to those of popular <inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula>-based approaches. We illustrate that our approach can handle problem instances with <inline-formula> <tex-math notation="LaTeX">$p =10,\!000$ </tex-math></inline-formula> features with certifiable optimality making it a highly scalable combinatorial variable selection approach in sparse linear modeling.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making (2019, background): “Our paper also relates to a nascent stream of research that leverages mixed-integer programming (MIP) to address ML tasks for which heuristics were traditionally employed (Bertsimas, King, and Mazumder 2015; Lou et al. 2013b; Mazumder and Radchenko 2015; Bertsimas and Dunn 2017; Verwer and Zhang 201”
Designing Fair, Efficient, and Interpretable Policies for Prioritizing Homeless Youth for Housing Resources (2018, methodology): “Our work is also related to recent applications of mixed-integer programming to machine learning [3, 17, 18].”