HOGgles: Visualizing Object Detection Features

By Carl Vondrick, Aditya Khosla, Tomasz Malisiewicz, Antonio Torralba. 2013

321 citations recorded by the source corpus.

We introduce algorithms to visualize feature spaces used by object detectors. The tools in this paper allow a human to put on 'HOG goggles' and perceive the visual world as a HOG based object detector sees it. We found that these visualizations allow us to analyze object detection systems in new ways and gain new insight into the detector's failures. For example, when we visualize the features for high scoring false alarms, we discovered that, although they are clearly wrong in image space, they do look deceptively similar to true positives in feature space. This result suggests that many of these false alarms are caused by our choice of feature space, and indicates that creating a better learning algorithm or building bigger datasets is unlikely to correct these errors. By visualizing feature spaces, we can gain a more intuitive understanding of our detection systems.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation (2013, background): “One advantage of simpler HOG-like features is that it’s easier to understand the information they carry (although [39] shows that with high-dimensional HOG features our intuition can fail us).”
Perceptual Losses for Real-Time Style Transfer and Super-Resolution (2016, methodology): “Mahendran and Vedaldi [7] invert features from convolutional networks by minimizing a feature reconstruction loss in order to understand the image information retained by different network layers; similar methods had previously been used to invert local binary descriptors [22,23] and HOG features [2”
Understanding deep image representations by inverting them (2014, methodology): “[33] for the visualisation of HOG features.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization (2016, positive): “We can also see that seemingly unreasonable predictions have reasonable explanations, an observation also made in HOGgles [44].”
Inverting Visual Representations with Convolutional Networks (2015, methodology): “Figure 2 shows an example image, its HOG representation, the results of inversion with existing methods [24, 19] and with our approach.”
Understanding Neural Networks through Representation Erasure (2016, methodology): “Methods for interpreting and visualizing neural models have also been significantly explored in vision (Vondrick et al., 2013; Vedaldi et al., 2014; Zeiler and Fergus, 2014; Weinzaepfel et al., 2011; Erhan et al., 2009; Simonyan et al., 2013; Klöppel et al., 2008), which we do not describe here…”
A Survey of Methods for Explaining Black Box Models (2018, background): “elated to producing a certain prediction by realizing a sort of articial image prototype. There is a signicant amount of work in understanding the representation of DNN by means of artifact images, [41,100,105,111]. We conclude this section presenting how [25] deals with artifacts in DNNs. Finding a”
Grad-CAM: Why did you say that? (2016, positive): “We can also see that seemingly unreasonable predictions have reasonable explanations, which is a similar observation to HOGgles [10].”
Understanding Image Representations by Measuring Their Equivariance and Equivalence (2014, methodology): “The first experiment is qualitative and uses the HOGgle technique (Vondrick et al. 2013) to visualize the transformed features.”
Visualizing Deep Convolutional Neural Networks Using Natural Pre-images (2015, methodology): “early neural networks in Williams (1986), Linden and Kindermann (1989), Lee and Kil (1994), and Lu et al.”
Context as Supervisory Signal: Discovering Objects with Predictable Context (2014, methodology): “only from the images on the left and the element-level correspondence shown by the red bounding boxes, we can synthesize a new, plausible car in HOG space, which we then render using inverse HOG [33].”
Watch and learn: Semi-supervised learning of object detectors from videos (2015, methodology): “es (see Figure3), we rely on the principle of decorrelated errors (similar to multi-view SSL [43,47]). The intuition is that the detector makes mistakes that are related to its feature representation [52], and a different feature representation would lead to different errors. Thus, if the errors in ”
Learning Aligned Cross-Modal Representations from Weakly Aligned Data (2016, unknown): “The motivation is to gain some visual understanding of which concepts are preserved across modalities and which information is discarded [39].”
Inverting Convolutional Networks with Convolutional Networks (2015, methodology): “There has been research on inverting various traditional computer vision representations: HOG [8], DSIFT [9], Local Binary Descriptors [10], Bag-of-Visual-Words [11].”
Privacy Preserving Image Queries for Camera Localization (2019, methodology): “Subsequently, others tried inverting and interpreting HOG features [70], bag of visual words features [29], and CNN features [41, 77, 78].”
Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262 (2018, background): “, [77], [82]) or extracting a transparent approximation such as a decision tree (e.”
Understanding classifier errors by examining influential neighbors (2015, background): “A few recent approaches [29, 32] have been developed for visualizing feature representations.”
Learning visual biases from human imagination (2014, methodology): “We sampled 150, 000 points from a standard normal multivariate distribution, and inverted each sample with the feature inversion algorithm from HOGgles [35].”