Enumerate Lasso Solutions for Feature Selection

By Satoshi Hara, Takanori Maehara. 2017

41 citations recorded by the source corpus.

We propose an algorithm for enumerating solutions to the Lasso regression problem.In ordinary Lasso regression, one global optimum is obtained and the resulting features are interpreted as task-relevant features.However, this can overlook possibly relevant features not selected by the Lasso.With the proposed method, we can enumerate many possible feature sets for human inspection, thus recording all the important features.We prove that by enumerating solutions, we can recover a true feature set exactly under less restrictive conditions compared with the ordinary Lasso.We confirm our theoretical results also in numerical simulations.Finally, in the gene expression and the text data, we demonstrate that the proposed method can enumerate a wide variety of meaningful feature sets, which are overlooked by the global optima.

Subfields: Artificial Intelligence, Molecular Biology, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Fairwashing: the risk of rationalization (2019, methodology): “To implement Algorithm 1, we modiﬁed the Lasso enumeration algorithm 2 of [18]. lllllllllllll llllll lll l llll l l llllllllllllllll llll l llllllllll lll l llllllllllllll llllllllllllllllllllll llll llll llllllllllll l l l ll lllllll ll l llll lllll llllll lllllllllll lllllllll lll llllllll lll ll…”
Approximate and Exact Enumeration of Rule Models (2018, methodology): “The approximate enumeration algorithm is summarized in Algorithm 1, which is an application of our previous al-gorithm for sparse linear models (Hara and Maehara 2017) to rule models.”
Feature Selection (2016, background): “ select featureswhosecorrespondingfeatureweightsare large,which motivates a surge of ℓ1-norm regularized feature selection methods [Zhu et al. 2004; Xu et al. 2014; Wei et al. 2016a; Wei and Yu 2016; Hara and Maehara 2017]. Also, the sparse vector w enables the ranking of features. Normally, the hig”