Learning Open-World Object Proposals without Learning to Classify

By Dahun Kim, Tsung-Yi Lin, Anelia Angelova, In-So Kweon, Weicheng Kuo. 2021

163 citations recorded by the source corpus.

Object proposals have become an integral pre-processing step of many vision pipelines including object detection, weakly supervised detection, object discovery, tracking, etc. Compared to the learning-free methods, learning-based proposals have become popular recently due to the growing interest in object detection. The common paradigm is to learn object proposals from data labeled with a set of object regions and their corresponding categories. However, this approach often struggles with novel objects in the open world that are absent in the training set. In this paper, we identify that the problem is that the binary classifiers in existing proposal methods tend to overfit to the training categories. Therefore, we propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlap with any ground-truth object (e.g., centerness and IoU). This strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization on COCO. We further explore more challenging cross-dataset generalization onto RoboNet and EpicKitchens dataset and demonstrate clear improvement over the state-of-the-art object detectors and object proposers. The code is publicly available at https://github.com/mcahny/object_localization_network.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Grounded Language-Image Pre-training (2021, background): “A line of work identiﬁes building a open-world object proposal module that could propose any novel object at test time as the key challenge [22, 25, 60, 74, 75]; GLIP offers a new perspective: the model does not need to propose every possible novel objects from an open set; rather it only needs to…”
Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, methodology): “7% improvements when combined with OLN (see Table 6 in section 5).”
Fast Segment Anything (2023, methodology): “The scores of competing methods are taken from [17], which test object proposal methods on all 80 COCO classes.”
Detecting Twenty-thousand Classes using Image-level Supervision (2022, background): “Further improvements to RPNs [17, 25, 27, 38] can hopefully lead to better results.”
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models (2022, methodology): “Inspired by the learned objectness (Kim et al., 2022; Kuo et al., 2015; Wang et al., 2020), we use class-agnostic box regression and mask prediction heads instead.”
Transformer-Based Visual Segmentation: A Survey (2023, unknown): “idea of region proposal network [111], [289] to generate class-agnostic mask proposals via different approaches, including anchor-based and query-based decoders in Sec.”
VOS: Learning What You Don't Know by Virtual Outlier Synthesis (2022, background): “Open-world object detection includes out-of-domain generalization (Kim et al., 2021; Wang et al., 2021), zero-shot object detection (Gu et al.”
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment (2023, background): “, which can be utilized to generate high-quality region proposals with good generalization [19, 26].”
Open-Vocabulary DETR with Conditional Matching (2022, background): “This problem has also been identified by a recent study [17], which suggests the binary nature of the region proposal network (RPN) could easily lead to overfitting to seen classes (thus fail to generalize to novel classes).”