Theoretical Comparison between the Gini Index and Information Gain Criteria

By Laura Elena Raileanu, Kilian Stoffel. 2004

638 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Information Systems, Statistical Learning Theory, Information Theory, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Information Systems

In Statistical Learning Theory

In Information Theory

In Signal Processing

Citations in the corpus

Measuring Data (2022, methodology): “…economics, sociology, and psychology, this measure has been applied within Machine Learning for classiﬁcation decision trees to select data splits (Raileanu and Stoffel, 2004; Strobl et al., 2007), similar to the usage of KL-Divergence discussed in subsection 4.1, although it remains…”
Survey on Automated Machine Learning (2019, background): “Common impurity functions are information gain from information theory or gini impurity (Raileanu and Stoffel, 2004).”
XOC: Explainable Observer-Classifier for Explainable Binary Decisions (2019, methodology): “We use the Gini impurity index as splitting criterion because it has a slight computational advantage over entropy-based methods (Raileanu & Stoffel, 2004), such as information gain.”
Towards the Deployment of Machine Learning Solutions in Network Traffic Classification: A Systematic Survey (2019, methodology): “Other techniques categorized as filter are the Gini Index [17], the Information Gain [18], the Laplacian Score and the Sparsity Scores [19], [20], among others.”
Big data for cyber physical systems in industry 4.0: a survey (2019, background): “The three popular functions are information gain (Quinlan 1986), gain ratio (Karegowda, Manjunath, and Jayaram 2010), and gini index (Raileanu and Sto ﬀ el 2004).”
An unsupervised feature selection algorithm based on ant colony optimization (2014, methodology): “There are numerous well-known univariate filter methods, such as information gain (Raileanu and Stoffel, 2004; Yu and Liu, 2003), gain ratio (Mitchell, 1997; Quinlan, 1986), symmetrical uncertainty (Biesiada and Duch, 2007; Yu and Liu, 2003), Gini index (Raileanu and Stoffel, 2004; Shang et al., 200”
Classification Based on Decision Tree Algorithm for Machine Learning (2021, background): “This intuitively informs how much knowledge of a random variable's value [57, 58].”
Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization (2010, methodology): “We compare our feature selection method (called as RFS) to several popularly used feature selection methods in bioinformatics, such as F -statistic [4], reliefF [11, 13], mRMR [19], t-test, and information gain [21].”
Machine learning methods in chemoinformatics (2014, methodology): “Each split within a tree is created based on the best partitioning that is possible according to the Gini criterion, 40 using any single-valued attribute from a randomly chosen subset of descriptors.”
Integration strategies of multi-omics data for machine learning analysis (2021, background): “They can be based on correlation (e.g., CFS [45], RCA [46]), distance (e.g., ReliefF [47]) or information gain [48] (e.g., mRMR [49]).”
Learning Decision Trees Recurrently Through Communication (2021, methodology): “Both DT and aDT are learned using the CART algorithm [3] and the Gini impurity index as splitting criterion due to its computational advantage over entropy-based methods [49].”