Bidirectional Graph Reasoning Network for Panoptic Segmentation

By Yangxin Wu, Gengwei Zhang, Yiming Gao, Xiajun Deng, Ke Gong, Xiaodan Liang, Liang Lin. 2020

60 citations recorded by the source corpus.

Recent researches on panoptic segmentation resort to a single end-to-end network to combine the tasks of instance segmentation and semantic segmentation. However, prior models only unified the two related tasks at the architectural level via a multi-branch scheme or revealed the underlying correlation between them by unidirectional feature fusion, which disregards the explicit semantic and co-occurrence relations among objects and background. Inspired by the fact that context information is critical to recognize and localize the objects, and inclusive object details are significant to parse the background scene, we thus investigate on explicitly modeling the correlations between object and background to achieve a holistic understanding of an image in the panoptic segmentation task. We introduce a Bidirectional Graph Reasoning Network (BGRNet), which incorporates graph structure into the conventional panoptic segmentation network to mine the intra-modular and inter-modular relations within and between foreground things and background stuff classes. In particular, BGRNet first constructs image-specific graphs in both instance and semantic segmentation branches that enable flexible reasoning at the proposal level and class level, respectively. To establish the correlations between separate branches and fully leverage the complementary relations between things and stuff, we propose a Bidirectional Graph Connection Module to diffuse information across branches in a learnable fashion. Experimental results demonstrate the superiority of our BGRNet that achieves the new state-of-the-art performance on challenging COCO and ADE20K panoptic segmentation benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yunchao Wei
2. Yi Yang
3. Huiyu Wang
4. Liang-Chieh Chen
5. Siyuan Qiao
6. Guoliang Kang
7. Alan L. Yuille
8. Gengwei Zhang

In Computer Vision and Pattern Recognition

1. Yi Yang
2. Yunchao Wei
3. Alan L. Yuille
4. Luc Van Gool
5. Dacheng Tao
6. Liang-Chieh Chen
7. Huiyu Wang
8. Jiashi Feng

In Graph Learning

1. Liang-Chieh Chen
2. Yi Yang

Citations in the corpus

• Scaling Wide Residual Networks for Panoptic Segmentation (2020, methodology): “Top-down approaches [43, 67, 50, 54, 60, 94, 51, 10, 47, 91] typically pair Mask R-CNN [27] with a light-weight ‘stuff‘ segmentation branch, while bottom-up approaches [95, 20, 87, 12, 88] group ‘thing‘ pixels from semantic segmentation predictions.”
• Few-Shot Segmentation via Cycle-Consistent Transformer (2021, methodology): “Besides, [43, 34] attempt to use graph attention networks [33, 40] to utilize all foreground support pixel features.”
• Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation (2020, unknown): “Most previous works [23, 40, 38] on panoptic segmentation share a similar architecture for semantic segmentation, i.”