Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance

By Xuan Vinh Nguyen, Julien Epps, James Bailey. 2010

2,394 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

On the Binding Problem in Artificial Neural Networks (2020, methodology): “For example, when a pixel-level segmentation is produced as part of segregation, then metrics such as AMI (Vinh et al., 2010) can be used to compare against the ground-truth.”
Tagger: Deep Unsupervised Perceptual Grouping (2016, methodology): “We evaluate the trained models using two metrics: First, the denoising cost on the validation set, and second we evaluate the segmentation into objects using the adjusted mutual information (AMI) score [35] and ignore the background and overlap regions in the Shapes dataset (consistent with Greff et”
Making AI Forget You: Data Deletion in Machine Learning (2019, background): “Normalized Mutual Information (NMI) [87, 49]: This quantity measures the agreement of the assigned clusters to the ground-truth labels, up to permutation.”
Towards Visually Grounded Sub-word Speech Unit Discovery (2019, methodology): “We quantify this by clustering the peak vectors using K-means, and computing the adjusted mutual information (AMI) [31] between the clustering output and the underlying phone and manner class sequences for each peak (Figure 6).”
Automated Machine Learning: Methods, Systems, Challenges (2019, neutral): “Certainly, not every cause-effect relation can be described by an additive noise model, but if an additive noise model fits in either direction, it is unlikely not to be the causal direction, as argued via algorithmic information theory in [51].”
A general species delimitation method with applications to phylogenetic placements (2013, methodology): “The data (Cox et al. , 2010) comprises four mitochondrial genes (cyt b, COI, 12S, 16S) and a total of 90 sequences (76 representing Gallotia and 14 outgroup sequences).”
Evaluating User Privacy in Bitcoin (2013, background): “NMI assesses the similarity of two groupings of the same items (in our case, Eprof and GTprof ), and takes higher values (1) the more identical the groupings are [19, 20].”
Causal Discovery Toolbox: Uncover causal relationships in Python (2019, methodology): “They rely on statistical tests, e.g. Pearson’s correlation or mutual information scores (Vinh et al., 2010).”
A Metric Learning Reality Check (2020, background): “Adjusted Mutual Information [55] removes this flaw, but still requires clustering to be done first.”
Self-labelling via simultaneous clustering and representation learning (2019, methodology): “A.1, we report standard clustering metrics (see (Vinh et al., 2010) for detailed deﬁnitions) of our trained models with regards to the ImageNet validation set ground-truth labels.”
Emergent Communication of Generalizations (2021, neutral): “(2) Here, E(I(M,C)) is evaluated with respect to a hypergeometric model of randomness, where M and C are assumed to be random permutations subject to the number of unique values in either set [39].”
Efficient image dataset classification difficulty estimation for predicting deep-learning accuracy (2018, methodology): “We assess the following known scores: adjusted mutual information [52], adjusted rand index [25], completeness, homogeneity, and the v-measure [42].”
Recurrent Ladder Networks (2017, methodology): “Following [13], the segmentation accuracy was computed using the adjusted mutual information (AMI) score [29] which is the mutual information between the ground truth segmentation and the estimated segmentation ⇡k scaled to give one when the segmentations are identical and zero when the output segme”
Analyzing Assumptions in Conversation Disentanglement Research Through the Lens of a New Dataset and Model (2018, neutral): “Adjusted Mutual Information (AMI, Vinh et al., 2010): A variation of mutual information that is corrected for random agreement by rescaling using the expected mutual information.”