Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-Supervised Object and Action Localization

By Krishna Kumar Singh, Yong Jae Lee. 2017

739 citations recorded by the source corpus.

We propose ‘Hide-and-Seek’, a weakly-supervised framework that aims to improve object localization in images and action localization in videos. Most existing weakly-supervised methods localize only the most discriminative parts of an object rather than all relevant parts, which leads to suboptimal performance. Our key idea is to hide patches in a training image randomly, forcing the network to seek other relevant parts when the most discriminative part is hidden. Our approach only needs to modify the input image and can work with any network designed for object localization. During testing, we do not need to hide any patches. Our Hide-and-Seek approach obtains superior performance compared to previous methods for weakly-supervised object localization on the ILSVRC dataset. We also demonstrate that our framework can be easily extended to weakly-supervised action localization.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “Object localization methods [32, 2] also utilize the regional dropout techniques for improving the localization ability of CNNs.”
Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, background): “Previous weakly-supervised approaches utilised video-level or transcript supervision, where the set [64,65,66,67,68,69] or sequence [70,71,72,73,74,75] of actions in the video are used in training, without temporal bounds.”
On Feature Normalization and Data Augmentation (2020, methodology): “Many subsequent papers proposed alternative flavors of this augmentation approach based on similar insights [4, 5, 7, 27, 28, 36, 51, 63, 70].”
Adversarial Complementary Learning for Weakly Supervised Object Localization (2018, methodology): “[35] proposed a method to augment the input images by randomly hiding patches so as to look for more object regions.”
Self-produced Guidance for Weakly-supervised Object Localization (2018, methodology): “…51.46 Backprop on GoogLeNet [41] 61.31 50.55 AlexNet-GAP [1] 67.19 52.16 VGGnet-GAP [1] 57.20 45.14 GoogLeNet-GAP [1] 56.40 43.00 GoogLeNet-HaS-32 [2] 54.53 - VGGnet-ACoL [20] 54.17 40.57 GoogLeNet-ACoL [20] 53 baseline methods on the ILSVRC 2016 dataset, as it has more than 1.2 million images of…”
Action Recognition From Single Timestamp Supervision in Untrimmed Videos (2019, background): “When only a few different actions are present in an untrimmed video, video-level supervision can prove sufficient to learn the actions even from long videos, as recently shown in [32, 22, 31, 28, 23].”
Weakly Supervised Object Localization and Detection: A Survey (2021, neutral): “On the other hand, [105], [129], [133], [162] develop methods to localize temporal actions in the given untrimmed videos, where the main goal is to predict the temporal boundary of each action instance contained in the weakly labeled training videos.”
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders (2020, methodology): “New Label Generation: To produce diverse predictions, we follow [49] and generate diverse annotations for the training dataset.”
Self-Challenging Improves Cross-Domain Generalization (2020, background): “Cutout [6] and HaS [24] randomly drop patches of input images.”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, methodology): “Recently, a lot of WSOL methods [59], [60], [1], [8], [60], [61], [62], [63] utilizing attention methods have been proposed, such as CAM [1], Grad-CAM [2], and ACoL [8].”
Understanding Deep Networks via Extremal Perturbations and Smooth Masks (2019, background): “Other works have leveraged perturbations at the input [23, 29] and intermediate layers [28] to perform weakly and fully supervised localization.”
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling (2021, background): “The “GT-known” [57,75,76,15] setting aims to localize a given ground-truth class.”
Radiologist-Level COVID-19 Detection Using CT Scans with Detail-Oriented Capsule Networks (2020, methodology): “It is inspired by the Hide and Seek [33] mechanism in which image patches are randomly dropped to encourage the network to look for other relevant parts.”
A flexible model for training action localization with varying levels of supervision (2018, methodology): “The UntrimmedNet [44] and Hide-and-Seek [38] methods are the current state of the art.”
3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization (2019, background): “The work of [25, 14, 28] use action category labels in videos for temporal localization, whereas [13] uses point-level supervision to spatio-temporally localize the actions.”