Contrastive Clustering

By Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, Xi Peng. AAAI Conference on Artificial Intelligence, 2021

831 citations recorded by the source corpus.

In this paper, we propose an online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. To be specific, for a given dataset, the positive and negative instance pairs are constructed through data augmentations and then projected into a feature space. Therein, the instance- and cluster-level contrastive learning are respectively conducted in the row and column space by maximizing the similarities of positive pairs while minimizing those of negative ones. Our key observation is that the rows of the feature matrix could be regarded as soft labels of instances, and accordingly the columns could be further regarded as cluster representations. By simultaneously optimizing the instance- and cluster-level contrastive loss, the model jointly learns representations and cluster assignments in an end-to-end manner. Besides, the proposed method could timely compute the cluster assignment for each individual, even when the data is presented in streams. Extensive experimental results show that CC remarkably outperforms 17 competitive clustering methods on six challenging image benchmarks. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline. The code is available at https://github.com/XLearning-SCU/2021-AAAI-CC.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d52ee822014835fcd5a4992c94579d9f95673d55

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Multi-view Contrastive Graph Clustering (2021, positive): “Contrastive clustering develops a dual contrastive learning framework, which conducts contrastive learning at instance-level as well as cluster-level [Li et al., 2021].”
Unsupervised Semantic Segmentation by Distilling Feature Correspondences (2022, background): “Clustering Unlike the linear probe, the clustering step does not have access to ground truth supervised labels.”
Self-Supervised Learning in Remote Sensing: A review (2022, background): “In addition, some recent works presented other efforts toward bridging contrastive learning and clustering, such as Prototypical Contrastive Learning (PCL) [82], Jigsaw Clustering [126], and Contrastive Clustering (CC) [205].”
Multi-level Feature Learning for Contrastive Multi-view Clustering (2021, positive): “For example, a one-stage online image clustering method was proposed in [19], which explicitly conducted contrastive learning in the instance-level and cluster-level.”
Dual Contrastive Prediction for Incomplete Multi-View Representation Learning (2022, background): “As one of the most effective unsupervised learning paradigms, contrastive learning [32], [33], [34], [35], [36], [37], [38], [39] has made huge developments in the field of representation learning.”
SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records (2021, background): “The majority of contrastive learning literature adopted self-supervised techniques [22], [23], [31], [32] by building augmented data with pseudo-labels.”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, background): “Further studies [198], [199], [200], [201], [202] found that joint category-level and instance-level contrastive learning (i.”
COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (2021, background): “As one of most effective unsupervised learning paradigms, contrastive learning [2, 4, 8, 23, 28, 30, 37, 38] has achieved state-of-the-art performance in representation learning.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, methodology): “Consistency regularization is also adopted by several deep clustering methods [149], [157], conforming its generic efﬁcacy beyond SSL. Lastly, discriminative unsupervised learning can also be conducted at both instance-level and group-level as in [150], [159].”