Local Aggregation for Unsupervised Learning of Visual Embeddings

By Chengxu Zhuang, Alex Zhai, Daniel L. K. Yamins. 2019

465 citations recorded by the source corpus.

Unsupervised approaches to learning in neural networks are of substantial interest for furthering artificial intelligence, both because they would enable the training of networks without the need for large numbers of expensive annotations, and because they would be better models of the kind of general-purpose learning deployed by humans. However, unsupervised networks have long lagged behind the performance of their supervised counterparts, especially in the domain of large-scale visual recognition. Recent developments in training deep convolutional embeddings to maximize non-parametric instance separation and clustering objectives have shown promise in closing this gap. Here, we describe a method that trains an embedding function to maximize a metric of local aggregation, causing similar data instances to move together in the embedding space, while allowing dissimilar instances to separate. This aggregation metric is dynamic, allowing soft clusters of different scales to emerge. We evaluate our procedure on several large-scale visual recognition datasets, achieving state-of-the-art unsupervised transfer learning performance on object recognition in ImageNet, scene recognition in Places 205, and object detection in PASCAL VOC.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

A Simple Framework for Contrastive Learning of Visual Representations (2020, neutral): “Table 6 compares our results with previous approaches (Zhuang et al., 2019; He et al., 2019a; Misra & van der Maaten, 2019; Hénaff et al., 2019; Kolesnikov et al., 2019; Donahue & Simonyan, 2019; Bachman et al., 2019; Tian et al., 2019) in the linear evaluation setting.”
Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “Contrastive learning is at the core of several recent works on unsupervised learning [61, 46, 36, 66, 35, 56, 2], which we elaborate on later in context (Sec.”
Are we done with ImageNet? (2020, neutral): “architecture design [2, 3, 4] or optimization [5] has yielded corresponding gains on other tasks and modalities [6, 7, 8]. Similarly, successful representation learning techniques applied to ImageNet [9, 10, 11] have yielded corresponding gains elsewhere [12, 13, 14, 15]. As recent results continue ”
Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “Prior work has explored diverse self-supervision cues such as predicting the spatialcontext [12], colorization [71], equivariance to transformations [18, 44]; alongside unsupervised techniques such as clustering [6, 72], generative modelling [14, 32], and exemplar learning [15].”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, methodology): “This model outperforms methods which use the same architecture, as well as many recent approaches which at times use substantially larger ones (Doersch & Zisserman, 2017; van den Oord et al., 2018; Kolesnikov et al., 2019; Zhuang et al., 2019; Donahue & Simonyan, 2019).”
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, background): “[98] Chengxu Zhuang, Alex Lin Zhai, and Daniel Yamins.”
Contrastive Multiview Coding (2019, background): “Concurrently with our work new papers have appeared on arXiv in the last few weeks: Deep Infomax++[3], CPC++[22] and [64].”
Emerging Properties in Self-Supervised Vision Transformers (2021, neutral): “Several variants allow automatic grouping of instances in the form of clustering [2, 7, 8, 21, 29, 35, 61, 65, 69].”
Self-Supervised Learning of Pretext-Invariant Representations (2019, neutral): “5 – LA [89] 25.”
What makes for good views for contrastive learning (2020, background): “For example: luminance and chrominance decomposition [53], randomly augmenting an image twice [58, 7, 4, 23, 60, 50, 65, 67], using different time-steps of videos [42, 66, 46, 22, 21], patches of the same image [27, 42, 26], multiple sensory data [39, 9, 44], or text and its context [37, 59, 35, 32]”
Prototypical Contrastive Learning of Unsupervised Representations (2020, methodology): “At the core of state-of-the-art unsupervised representation learning algorithms [1, 2, 3, 4, 11, 5, 6, 7, 9], instance-wise contrastive learning aims to learn an embedding space where samples (e.”
Self-labelling via simultaneous clustering and representation learning (2019, neutral): “∗, (Zhuang et al., 2019) 18.”
Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations (2021, background): “Many works use a memory bank to store the instance class representation vector [67, 77, 59, 23, 44, 9], while some others explore the usage of in-batch negative samples instead [14, 71, 28, 31].”
Audio-Visual Instance Discrimination with Cross-Modal Agreement (2020, background): “Contrastive Learning techniques use a contrastive loss [28] to learn representations either by predicting parts of the data [32, 33, 61], or discriminating between individual training instances [17, 19, 31, 34, 53, 86, 88, 94].”
Unsupervised Learning From Video With Deep Neural Embeddings (2019, methodology): “They are based on a simple probabilistic perspective for interpreting compact embedding spaces [37, 40].”