Representation Learning by Learning to Count

By Mehdi Noroozi, Hamed Pirsiavash, Paolo Favaro. 2017

379 citations recorded by the source corpus.

We introduce a novel method for representation learning that uses an artificial supervision signal based on counting visual primitives. This supervision signal is obtained from an equivariance relation, which does not require any manual annotation. We relate transformations of images to transformations of the representations. More specifically, we look for the representation that satisfies such relation rather than the transformations that match a given representation. In this paper, we use two image transformations in the context of counting: scaling and tiling. The first transformation exploits the fact that the number of visual primitives should be invariant to scale. The second transformation allows us to equate the total number of visual primitives in each tile to that in the whole image. These two transformations are combined in one constraint and used to train a neural network with a contrastive loss. The proposed task produces representations that perform on par or exceed the state of the art in transfer learning benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “Prior work has explored diverse self-supervision cues such as predicting the spatialcontext [12], colorization [71], equivariance to transformations [18, 44]; alongside unsupervised techniques such as clustering [6, 72], generative modelling [14, 32], and exemplar learning [15].”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “tives formulate prediction in terms of missing pixel-level details such as context or color [14,69,46,38,64], another class of methods deﬁnes prediction in higher-level, learned representation spaces [25,16,47,49,51]. These latter techniques are appealing because they may encourage the learning of a”
S4L: Self-Supervised Semi-Supervised Learning (2019, background): “Another example is [25], that enforces additivity constraint on visual representation: the sum of representations of all image patches should be close to representation of the whole image.”
Contrastive Multiview Coding (2019, methodology): “As many previous unsupervised methods are evaluated with AlexNet [43] on ImageNet [16, 42, 17, 84, 54, 18, 85, 55, 22, 11, 83], we also include the the results of CMC using this network.”
Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018, methodology): “To learn a representation of images, the tasks could be: predicting the context [2], counting the objects [28], filling in missing parts of an image [31], recovering colors from grayscale images [47], or even solving a jigsaw puzzle [27].”
Unsupervised Representation Learning by Predicting Image Rotations (2018, background): “Speciﬁcally, we ﬁrst train a RotNet model on the training images of the ImageNet dataset and then we evaluate the performance of the self-supervised features on the image Context (Doersch et al., 2015) 19.7 26.7 31.9 32.7 30.9 Context Encoders (Pathak et al., 2016b) 18.2 23.2 23.4 21.9 18.4 Coloriza”
A critical analysis of self-supervision, or what we can learn from a single image (2019, neutral): “…(Caron et al., 2018), noise-as-targets (Bojanowski & Joulin, 2017), colorization (Zhang et al., 2016; Larsson et al., 2017), generation (Jenni & Favaro, 2018; Ren & Lee, 2018; Donahue et al., 2017), geometry (Dosovitskiy et al., 2016; Gidaris et al., 2018) and counting (Noroozi et al., 2017).”
Local Aggregation for Unsupervised Learning of Visual Embeddings (2019, background): “Over many decades of work, substantial progress has been achieved using a wide variety of unsupervised learning approaches [7, 72, 76, 40, 14, 16, 75, 52, 68, 53].”
Revisiting Self-Supervised Visual Representation Learning (2019, background): “propose an equivariance relation to match the sum of multiple tiled representations to a single scaled representation [32].”
ScaleNet: An Unsupervised Representation Learning Method for Limited Information (2023, methodology): “The ScaleNet is examined by training on the ImageNet dataset similar to other self-supervised learning methods [14,37,38].”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, methodology): “We briefly describe the two image based self-supervised approaches [49, 74] that we study in this work and refer the reader to the original papers for detailed explanations.”
Deep Clustering for Unsupervised Learning of Visual Features (2018, background): “Appart from spatial and temporal coherence, many other signals have been explored: image colorization [33, 71], cross-channel prediction [72], sound [44] or instance counting [43].”
Self-labelling via simultaneous clustering and representation learning (2019, background): “…(Jenni & Favaro, 2018; Ren & Lee, 2018; Donahue et al., 2017; Donahue & Simonyan, 2019), predictive coding (Oord et al., 2018; Hénaff et al., 2019), geometry (Dosovitskiy et al., 2016), predicting transformations (Gidaris et al., 2018; Zhang et al., 2019) and counting (Noroozi et al., 2017).”
Taskonomy: Disentangling Task Transfer Learning (2018, background): “We review the topics most pertinent to vision within the constraints of space: Self-supervised learning methods leverage the inherent relationships between tasks to learn a desired expensive one (e.g. object detection) via a cheap surrogate (e.g. colorization) [68, 72, 17, 103, 100, 69].”
Boosting Self-Supervised Learning via Knowledge Transfer (2018, background): “Finally, recent work [23] has also exploited the link between relative transformations of images to define relative transformations of features.”
AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations Rather Than Data (2019, background): “The results show the AET models outperform the other unsupervised models in most of cases, except on Conv1 and Conv2, Counting [19] performs slightly better.”
Improvements to Context Based Self-Supervised Learning (2017, methodology): “A common remedy is to withhold color data from one or more channels by using channel-dropping [9, 10], channel replication [25], or conversion to gray scale [33].”
Self-Supervised Feature Learning by Learning to Spot Artifacts (2018, methodology): “[29] introduces counting of visual primitives as a pretext task for representation learning.”