LinkNet: Relational Embedding for Scene Graph

By Sanghyun Woo, Dahun Kim, In-So Kweon. 2018

158 citations recorded by the source corpus.

Objects and their relationships are critical contents for image understanding. A scene graph provides a structured description that captures these properties of an image. However, reasoning about the relationships between objects is very challenging and only a few recent works have attempted to solve the problem of generating a scene graph from an image. In this paper, we present a method that improves scene graph generation by explicitly modeling inter-dependency among the entire object instances. We design a simple and effective relational embedding module that enables our model to jointly represent connections among all related objects, rather than focus on an object in isolation. Our method significantly benefits the main part of the scene graph generation task: relationship classification. Using it on top of a basic Faster R-CNN, our model achieves state-of-the-art results on the Visual Genome benchmark. We further push the performance by introducing global context encoding module and geometrical layout encoding module. We validate our final model, LinkNet, through extensive ablation studies, demonstrating its efficacy in scene graph generation.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Yi Yang
3. Dacheng Tao
4. Xiaogang Wang
5. Trevor Darrell
6. Yue Cao
7. Han Hu
8. Ming-Hsuan Yang

Citations in the corpus

• Unbiased Scene Graph Generation From Biased Training (2020, critical): “Most of the existing methods [62, 60, 7, 26, 68, 53, 64, 10, 43, 59] struggle for better feature extraction networks.”
• Memory Enhanced Global-Local Aggregation for Video Object Detection (2020, methodology): “Like our method that tries to aggregate information beyond a small local range, [29, 20, 28, 33] share a similar spirit and shows superior results in different areas.”
• A Comprehensive Survey of Scene Graphs: Generation and Application (2021, methodology): “More specifically, LinkNet designs a simple and effective relational embedding module that jointly learns the connections between all related objects.”