GPS-Net: Graph Property Sensing Network for Scene Graph Generation
By Xin Lin, Changxing Ding, Jinquan Zeng, Dacheng Tao. Computer Vision and Pattern Recognition, 2020
338 citations recorded by the source corpus.
Scene graph generation (SGG) aims to detect objects in an image along with their pairwise relationships. There are three key properties of scene graph that have been underexplored in recent works: namely, the edge direction information, the difference in priority between nodes, and the long-tailed distribution of relationships. Accordingly, in this paper, we propose a Graph Property Sensing Network (GPS-Net) that fully explores these three properties for SGG. First, we propose a novel message passing module that augments the node feature with node-specific contextual information and encodes the edge direction information via a tri-linear model. Second, we introduce a node priority sensitive loss to reflect the difference in priority between nodes during training. This is achieved by designing a mapping function that adjusts the focusing parameter in the focal loss. Third, since the frequency of relationships is affected by the long-tailed distribution problem, we mitigate this issue by first softening the distribution and then enabling it to be adjusted for each subject-object pair according to their visual appearance. Systematic experiments demonstrate the effectiveness of the proposed techniques. Moreover, GPS-Net achieves state-of-the-art performance on three popular databases: VG, OI, and VRD by significant gains under various settings and metrics. The code and models are available at https://github.com/taksau/GPS-Net.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/56092013ab20394725719910fc7d4fe2274f3d54
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Xiaojun Chang
- Alexander G. Hauptmann
- Zhihui Li
- Chaoyue Wang
- Daqing Liu
- Xiaojiang Chen
- Pengzhen Ren
Citations in the corpus
- A Comprehensive Survey of Scene Graphs: Generation and Application (2021, methodology): “In contrast, VCTree [72], KERN [20], GPS-Net [185], GB-NET [122], in addition to using visual features, they also use other knowledge (for example, language embedding, statistical information, and counterfactual causality, etc.).”
- Multimodal Learning With Transformers: A Survey (2022, background): “to 3D scene [40], single image to 3D human texture estimation [223], RGB to scene graph [19], [224], [225], [226], graph to graph [33], knowledge graph to text [227], video to scene”
- Modeling Image Composition for Complex Scene Generation (2022, background): “Compared with other conditions for complex scene generation, including textual descriptions [27], scene graphs [13, 23], and segmentation masks [20], layouts are much more user-friendly, controllable and flexible [45].”