Information-based clustering.

By Noam Slonim, Gurinder S. Atwal, Gasper Tkacik, William Bialek. 2005

239 citations recorded by the source corpus.

In an age of increasingly large data sets, investigators in many different disciplines have turned to clustering as a tool for data analysis and exploration. Existing clustering methods, however, typically depend on several nontrivial assumptions about the structure of data. Here, we reformulate the clustering problem from an information theoretic perspective that avoids many of these assumptions. In particular, our formulation obviates the need for defining a cluster "prototype," does not require an a priori similarity metric, is invariant to changes in the representation of the data, and naturally captures nonlinear relations. We apply this approach to different domains and find that it consistently produces clusters that are more coherent than those extracted by existing algorithms. Finally, our approach provides a way of clustering based on collective notions of similarity rather than the traditional pairwise measures.

Subfields: Artificial Intelligence, Molecular Biology, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Statistical and Nonlinear Physics

Citations in the corpus

Deep Semi-Supervised Anomaly Detection (2019, methodology): “Examples where the Infomax principle has been applied include tasks such as independent component analysis (Bell & Sejnowski, 1995), clustering (Slonim et al., 2005; Ji et al., 2018), generative modeling (Chen et al.”
Inverse statistical problems: from the inverse Ising problem to data science (2017, methodology): “identified clusters in stocks using an informationbased metric of stock prices [206] .”
A universal framework for regulatory element discovery across all genomes and data types. (2007, methodology): “We clustered the � 6000 genes into 78 nonoverlapping clusters by using Iclust ( Slonim et al., 2005 ).”
Discriminative Clustering by Regularized Information Maximization (2010, methodology): “The method of [20] aims to maximize a similarity measure computed between members within the same cluster while penalizing the mutual information between the cluster label y and the input x.”