Relief-Based Feature Selection: Introduction and Review

By Ryan J. Urbanowicz, Melissa Meeker, William La Cava, Randal S. Olson, Jason H. Moore. Journal of Biomedical Informatics, 2017

1,180 citations recorded by the source corpus.

Feature selection plays a critical role in biomedical data mining, driven by increasing feature dimensionality in target problems and growing interest in advanced but computationally expensive methodologies able to model complex associations. Specifically, there is a need for feature selection methods that are computationally efficient, yet sensitive to complex patterns of association, e.g. interactions, so that informative features are not mistakenly eliminated prior to downstream modeling. This paper focuses on Relief-based algorithms (RBAs), a unique family of filter-style feature selection algorithms that have gained appeal by striking an effective balance between these objectives while flexibly adapting to various data characteristics, e.g. classification vs. regression. First, this work broadly examines types of feature selection and defines RBAs within that context. Next, we introduce the original Relief algorithm and associated concepts, emphasizing the intuition behind how it works, how feature weights generated by the algorithm can be interpreted, and why it is sensitive to feature interactions without evaluating combinations of features. Lastly, we include an expansive review of RBA methodological research beyond Relief and its popular descendant, ReliefF. In particular, we characterize branches of RBA research, and provide comparative summaries of RBA algorithms including contributions, strategies, functionality, time complexity, adaptation to key data characteristics, and software availability.

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/1fcb6d8d751c86a583dd704d222168225aff0eeb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Heart Disease Identification Method Using Machine Learning Classification in E-Healthcare (2020, neutral): “Relief [31] algorithm assigns weights to each data set features and updated weights automatically.”
A Review of Feature Selection Methods for Machine Learning-Based Disease Risk Prediction (2022, methodology): “Notably, RBAs can detect pair-wise feature interactions, some RBAs (e.g., ReliefF, MultiSURF) can even detect higher order ( > 2 way) interactions (Urbanowicz et al., 2018a).”
Flood Detection and Susceptibility Mapping Using Sentinel-1 Remote Sensing Data and a Machine Learning Approach: Hybrid Intelligence of Bagging Ensemble Based on K-Nearest Neighbor Classifier (2020, background): “Accordingly, an attribute is ranked highest if the same value is obtained for instances of the same class and distinguished for instances of different classes [127,128].”
A comprehensive survey on recent metaheuristics for feature selection (2022, methodology): “focused on Reliefbased algorithms (RBAs), which are filter-style feature selection algorithms [79].”
A Hybrid Intelligent System Framework for the Prediction of Heart Disease Using Machine Learning Algorithms (2018, background): “For each k, Rk is the “target” instance and the feature score vectorW is updated [23].”