Improved Deep Embedded Clustering with Local Structure Preservation

By Xifeng Guo, Long Gao, Xinwang Liu, Jianping Yin. International Joint Conference on Artificial Intelligence, 2017

1,034 citations recorded by the source corpus.

Deep clustering learns deep feature representations that favor clustering task using neural networks. Some pioneering work proposes to simultaneously learn embedded features and perform clustering by explicitly deﬁning a clustering oriented loss. Though promising performance has been demonstrated in various applications, we observe that a vital ingredient has been overlooked by these work that the deﬁned clustering loss may corrupt feature space, which leads to non-representative mean-ingless features and this in turn hurts clustering performance. To address this issue, in this paper, we propose the Improved Deep Embedded Clustering (IDEC) algorithm to take care of data structure preservation. Speciﬁcally, we manipulate feature space to scatter data points using a clustering loss as guidance. To constrain the manipulation and maintain the local structure of data generating distribution, an under-complete autoencoder is applied. By integrating the clustering loss and autoencoder’s reconstruction loss, IDEC can jointly optimize cluster labels assignment and learn features that are suitable for clustering with local structure preservation. The resultant optimization problem can be effectively solved by mini-batch stochastic gradient descent and backpropagation. Experiments on image and text datasets empirically validate the importance of local structure preservation and the effectiveness of our algorithm.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/49fd2cb67a7864168ff097aaeb62a38fcd6be7af

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Structural Deep Clustering Network (2020, methodology): “n ability of deep learning. Hence learning an effective data representation is a crucial prerequisite for deep clustering. For example, [27] uses the representation learned by autoencoder in K-means; [4, 26] leverage a clustering loss to help autoencoder learn the data representation with high clust”
Deep Graph Clustering via Dual Correlation Reduction (2021, methodology): “Three representative deep generative methods, i.e., AE (Yang et al. 2017), DEC (Xie, Girshick, and Farhadi 2016), and IDEC (Guo et al. 2017), train an auto-encoder and then perform a clustering algorithm over the learned latent embedding.”
NewsEmbed: Modeling News through Pre-trained Document Representations (2021, background): “They usually compute textual representation by features such as bag-of-words [1], learned topics [52], or deep learning representations [27].”
Clustering single-cell RNA-seq data with a model-based deep learning approach (2019, background): “Such a regularization term can help to prevent the deep embedding function from overfitting 31 .”
Contrastive Clustering (2021, background): “Benefit from the powerful representative ability of deep neural networks, deep clustering (Xie, Girshick, and Farhadi 2016; Guo et al. 2017a; Li et al. 2020) has shown promising performance on complex datasets.”
Self-supervised Context-aware Style Representation for Expressive Speech Synthesis (2022, positive): “…p ik is As this kind of clustering distorts the original space of representation and weakens the representation ability of implicit feature [18], we add an autoencoder structure with reconstruction loss as in Equation 8, in order to preserve local structure of feature space and avoid…”
Deep Clustering with Convolutional Autoencoders (2017, methodology): “IDEC [17] denotes the algorithm that adds reconstruction loss Lr to DEC’s objective.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, methodology): “In offline clustering, unsupervised training is alternated between a cluster assignment step and a network training step [139], [140], [170], [171], [182], [191], [192], [193].”
A Review of Speaker Diarization: Recent Advances with Deep Learning (2021, background): “The key idea is that each embedding has a probability of “belonging” to each of the available speaker cluster [150, 64],”
Deep Spectral Clustering Using Dual Autoencoder Network (2019, methodology): “…learning (JULE) [44], deep embedded regularized clustering (DEPICT) [10], improved deep embedded clustering with locality preservation (IDEC) [12], deep spectral clustering with a set of nearest neighbor pairs (SpectralNet) [36], clustering with GAN (ClusterGAN) [30] and GAN with the mutual…”