Feature subset selection in large dimensionality domains

By Iffat A. Gheyas, Leslie S. Smith. Pattern Recognition, 2010

616 citations recorded by the source corpus.

Searching for an optimal feature subset from a high dimensional feature space is known to be an NP-complete problem. We present a hybrid algorithm, SAGA, for this task. SAGA combines the ability to avoid being trapped in a local minimum of simulated annealing with the very high rate of convergence of the crossover operator of genetic algorithms, the strong local search ability of greedy algorithms and the high computational efficiency of generalized regression neural networks. We compare the performance over time of SAGA and well-known algorithms on synthetic and real datasets. The results show that SAGA outperforms existing algorithms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bde5538e7dbe75911ecf9e6253325906bee06685

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yaochu Jin
2. Ran Cheng
3. Mengjie Zhang
4. Bing Xue
5. Shenkai Gu
6. Xing-yi Zhang
7. Will N. Browne
8. Ye Tian

In Computer Vision and Pattern Recognition

1. Liangpei Zhang
2. Bo Du
3. Lefei Zhang
4. Dacheng Tao
5. Mengjie Zhang
6. Xin Huang
7. Bing Xue
8. Qian Zhang

Citations in the corpus

• Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding (2015, background): “provides more potential discriminative information to distinguish the patterns of different classes, the feature vector of each view is usually lies in a high dimensional feature space and thus combining them together frequently leads to the problem of ”curse of dimensionality” [3, 4, 5].”
• Survey on Feature Selection (2015, methodology): “Compared to filters, Gheyas and Smith (2010) stated that wrapper’s effectiveness comes at the expense of their computational cost.”
• Supervised hybrid feature selection based on PSO and rough sets for medical diagnosis (2014, methodology): “Searching for an optimal feature subset from a highdimensional feature space is an NP-complete problem [9]; hence, traditional optimization algorithms are inefficient in solving large-scale feature selection problems.”
• An improved particle swarm optimization for feature selection (2011, methodology): “Meanwhile, searching for an optimal feature subset in a high-dimensional feature space is an NP-complete problem [8]; hence, traditional optimization algorithms are inefficient when solving large-scale feature selection problems.”
• Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms (2014, background): “There could be two-way or multi-way interactions among features [1, 5].”
• An embedded feature selection method for imbalanced data classification (2019, background): “Backward elimination [18] is the opposite of forward selection.”
• The Perils of Ignoring Data Suitability - The Suitability of Data used to Train Neural Networks Deserves More Attention (2011, background): “Feature selection is another well studied field with many proposed techniques, see (Gheyas and Smith, 2010) for a recent example.”
• EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations (2019, background): “In recent years, there have been a series of methods proposed to remit the influence of multicollinearity [65], [71], [72].”
• Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach (2013, neutral): “they may even reduce the classification performance due to the large search space known as “the curse of dimensionality” [1].”
• Feature selection for high-dimensional classification using a competitive swarm optimizer (2016, background): “Unfortunately, irrelevant and redundant features will not only reduce the training efficiency, but also negatively influence the performance of machine learning thus trained with them, which is mainly caused by the curse of dimensionality (Gheyas and Smith 2010).”
• A Survey on Evolutionary Computation Approaches to Feature Selection (2016, methodology): “techniques [38], [53], [139], while most were applied individually to address feature selection problems [54], [56], [79], [151], [219]–[221], [223]–[225], [231].”
• Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues (2014, methodology): “Traditional dimension reduction methods can be generally classified into feature extraction (or feature transformation) and feature selection (or variables selection) [14], [15].”
• Supervised, Unsupervised, and Semi-Supervised Feature Selection: A Review on Gene Selection (2016, background): “According to Gheyas and Smith [10], a good search strategy should provide good global search capability, rapid convergence to near optimal solution, good local search ability, and high computational efficiency.”