Convolutional Clustering for Unsupervised Learning
By Aysegul Dundar, Jonghoon Jin, Eugenio Culurciello. 2015
85 citations recorded by the source corpus.
The task of labeling data for training deep neural networks is daunting and tedious, requiring millions of labels to achieve the current state-of-the-art results. Such reliance on large amounts of labeled data can be relaxed by exploiting hierarchical features via unsupervised learning techniques. In this work, we propose to train a deep convolutional network based on an enhanced version of the k-means clustering algorithm, which reduces the number of correlated parameters in the form of similar filters, and thus increases test categorization accuracy. We call our algorithm convolutional k-means clustering. We further show that learning the connection between the layers of a deep convolutional neural network improves its ability to be trained on a smaller amount of labeled data. Our experiments show that the proposed algorithm outperforms other techniques that learn filters unsupervised. Specifically, we obtained a test accuracy of 74.1% on STL-10 and a test error of 0.5% on MNIST.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Eldad Haber
- Lars Ruthotto
- Nicu Sebe
- Enver Sangineto
- Aliaksandr Siarohin
- Michael Pfeiffer
- Aleksandr Ermolov
- Thomas Pfeil
In Computer Vision and Pattern Recognition
- Aliaksandr Siarohin
- Nicu Sebe
- Enver Sangineto
- Eldad Haber
- Lars Ruthotto
- Michael Pfeiffer
- Aleksandr Ermolov
- Thomas Pfeil
In Representation & Self-Supervised Learning
- Aliaksandr Siarohin
- Michael Pfeiffer
- Eldad Haber
- Nicu Sebe
- Enver Sangineto
- Aleksandr Ermolov
- Thomas Pfeil
- Lars Ruthotto
Citations in the corpus
- Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces (2017, neutral): “A number of unsupervised algorithms have been proposed to that effect, such as sparse-coding, restricted Boltzmann machines (RBMs), auto-encoders [44], and K-means clustering [4, 8, 20].”
- Self-Supervised Feature Learning by Learning to Spot Artifacts (2018, neutral): “[10] 74.1% - Huang et al .”
- Steering Self-Supervised Feature Learning Beyond Local Pixel Statistics (2020, unknown): “We then fine-tuned the network [14] 74.1% Hjelm et al .”
- Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, result): “This is encouraging, since recent work (e.g., Coates et al., 2011; Dundar et al., 2015) has shown that competitive convolutional networks can be trained with unsupervised learning of filters.”
- Whitening for Self-Supervised Representation Learning (2020, methodology): “7 CPW2 66.29 86.94 14 Ermolov et al. Table 7. Fully and semi-supervised classication on STL10. a indicates results reported in [26]. yindicates fully supervised method. Method Accuracy Dundar et al. [13] a 74.1 Cutout [9] ya 87.3 Oyallon et al. [34] ya 87.6 DeepCluster [3] a 73.4 ADC [18] a 56.7 DIM”
- Deep Neural Networks Motivated by Partial Differential Equations (2018, result): “at the Hamiltonian network achieves the best test accuracy using only about half as many trainable weights as the other two networks. These results are competitive with the results reported, e.g., in [44, 14]. Fine-tuning of hyperparameters such as step size, number of time steps, and width of the n”