Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships

By Yong Liu, Ruiping Wang, Shiguang Shan, Xilin Chen. 2018

242 citations recorded by the source corpus.

Context is important for accurate visual recognition. In this work we propose an object detection algorithm that not only considers object visual appearance, but also makes use of two kinds of context including scene contextual information and object relationships within a single image. Therefore, object detection is regarded as both a cognition problem and a reasoning problem when leveraging these structured information. Specifically, this paper formulates object detection as a problem of graph structure inference, where given an image the objects are treated as nodes in a graph and relationships between the objects are modeled as edges in such graph. To this end, we present a so-called Structure Inference Network (SIN), a detector that incorporates into a typical detection framework (e.g. Faster R-CNN) with a graphical model which aims to infer object state. Comprehensive experiments on PASCAL VOC and MS COCO datasets indicate that scene context and object relationships truly improve the performance of object detection with more desirable and reasonable outputs.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “After that, lots of algorithms are proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multiscale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and”
OCNet: Object Context Network for Scene Parsing (2018, background): “Context has been investigated for both object detection [5, 17] and part detection [8].”
Graph-FCN for Image Semantic Segmentation (2019, methodology): “These methods model the detected object as the nodes of a graph in the image, and by extracting the relation between the objects to improve the detection accuracy[15].”
Language-Conditioned Graph Networks for Relational Reasoning (2019, methodology): “Graph networks have been successfully applied to various tasks, from object detection [25] and region clas-siﬁcation [7] to human-object interaction [30] and activity recognition [12].”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, methodology): “Method BIMEF CRM Dong LIME MF RRM SRIE Retinex-Net MSR NPE GLAD KinD MIRNet [116] [117] [29] [40] [34] [67] [34] [107] [54] [101] [102] [124] (Ours)”
Bilinear Graph Networks for Visual Question Answering (2019, background): “In image-based tasks, GNN gathers information from all the grids [20], [37] or proposals [38] other than surroundings whose size is limited by the receptive fields of convolution neural networks (CNNs), and it aggregates features over coordinate space to improve the performance of object detection a”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, methodology): “Method BIMEF CRM Dong LIME MF RRM SRIE Retinex-Net MSR NPE GLAD KinD KinD++ MIRNet-v2 [126] [127] [128] [129] [130] [131] [130] [85] [81] [132] [133] [4] [134] (Ours)”
Object Detection in 20 Years: A Survey (2019, unknown): “139], MultiPath [140], GBDNet [141, 142], CC-Net [143], MultiRegion-CNN [144], CoupleNet [145], DPM [14, 15], StructDet [146], YOLO [20], RFCN++ [147], ION [148], AttenContext [149], CtxSVM [150], PersonContext [151], SMN [152], RetinaNet [23], SIN [153].”
A Survey of Deep Learning-Based Object Detection (2019, methodology): “[70] adopt scene contextual information to further improve accuracy.”
Recent Advances in Deep Learning for Object Detection (2019, background): “proposed Struc- 1105 ture Inference Net (SIN) [137] which formulated object detection as 1106 a graph inference problem by considering scene contextual infor- 1107 mation and object relationships.”
Deep Learning for Generic Object Detection: A Survey (2018, background): “SIN [137] considered two kinds of context including scene contextual information and object relationships within a single image.”
Iterative Context-Aware Graph Inference for Visual Dialog (2020, background): “Graph neural networks have attracted attention in various tasks [32, 22, 17, 8, 26].”
General Multi-label Image Classification with Transformers (2020, background): “While a lot of work focuses on single-label classification [14], there is also an ample body of work on both multi-label prediction and exploiting label dependencies [13, 39, 35, 22].”
Hallucination In Object Detection — A Study In Visual Part VERIFICATION (2021, background): “Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image [14, 15] while also affected by scene context [16, 17, 18, 19, 20].”