Graph-Based Global Reasoning Networks

By Yunpeng Chen, Marcus Rohrbach, Zhicheng Yan, Shuicheng Yan, Jiashi Feng, Yannis Kalantidis. 2018

506 citations recorded by the source corpus.

Globally modeling and reasoning over relations between regions can be beneficial for many computer vision tasks on both images and videos. Convolutional Neural Networks (CNNs) excel at modeling local relations by convolution operations, but they are typically inefficient at capturing global relations between distant regions and require stacking multiple convolution layers. In this work, we propose a new approach for reasoning globally in which a set of features are globally aggregated over the coordinate space and then projected to an interaction space where relational reasoning can be efficiently computed. After reasoning, relation-aware features are distributed back to the original coordinate space for down-stream tasks. We further present a highly efficient instantiation of the proposed approach and introduce the Global Reasoning unit (GloRe unit) that implements the coordinate-interaction space mapping by weighted global pooling and weighted broadcasting, and the relation reasoning via graph convolution on a small graph in interaction space. The proposed GloRe unit is lightweight, end-to-end trainable and can be easily plugged into existing CNNs for a wide range of tasks. Extensive experiments show our GloRe unit can consistently boost the performance of state-of-the-art backbone architectures, including ResNet, ResNeXt, SE-Net and DPN, for both 2D and 3D CNNs, on image classification, semantic segmentation and video action recognition task.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Graph Learning

Citations in the corpus

Visual Transformers: Token-based Image Representation and Processing for Computer Vision (2020, methodology): “Graph convolutions in vision models: Our work is also related to previous efforts such as GloRe [6], LatentGNN [47], and [26] that densely relate concepts in latent space using graph convolutions.”
Object-Contextual Representations for Semantic Segmentation (2019, methodology): “Double Attention and its related work [8, 61, 9, 36, 34, 59, 32] and ACFNet [61] group the pixels into a set of regions, and then augment the pixel representations by aggregating the region representations with the consideration of their context relations predicted by using the pixel representation.”
Hierarchical Multi-Scale Attention for Semantic Segmentation (2020, background): “OCRNet [9], DANET [10], CFNet [11], OCNet [12] and other related work [13, 14, 15, 16, 17, 18, 19, 20] use such relationships to build better context.”
Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition (2021, unknown): “Considerable efforts have been made to design graph convolutional neural networks (GCN) for modelling graph-structured data (Kipf and Welling 2017; Chen et al. 2019; Wang et al. 2019).”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, background): “Meanwhile, another burgeoning research area for semantic segmentation is how to embed the dot-product attention into a Graph Convolutional Network (GCN) and optimize the complexity of the attention [79-83].”
Adaptively Connected Neural Networks (2019, methodology): “Similarly, PSANet [52] is built upon NLN by introducing a position encoding to each pixel; GloRe [3] improves on NLN in a way of using a graphCNN to capture the global dependencies.”
G-TAD: Sub-Graph Localization for Temporal Action Detection (2019, methodology): “Graphs have been widely used for data/feature representation in various video understanding tasks, such as action recognition [31, 47, 10], and action localization [55].”
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution (2019, methodology): “In this part, we compare OctConv with MG-Conv [25], GloRe [8], Elastic [43] and bL-Net [4] which share a similar idea as our method.”
Attentive Group Equivariant Convolutional Networks (2020, background): “…al., 2018; Park et al., 2018; Woo et al., 2018; Wang et al., 2018; Ilse et al., 2018; Hu et al., 2019; Ramachandran et al., 2019; Cao et al., 2019; Chen et al., 2019; Bello et al., 2019; Lin et al., 2019; Diaconu & Worrall, 2019; Romero & Hoogendoorn, 2020) exclusively utilize translation (or…”
Attention mechanisms in computer vision: A survey (2021, background): “GloRe [71] understands self-attention from a graph learning perspective.”
PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection (2019, methodology): “Thus for the DLA-based model, we concatenate the last three level features and apply a graph-based global reasoning module [3] to enlarge the receptive field for the interaction point and displacement prediction.”
Context-Gated Convolution (2019, methodology): “The previous work (Vaswani et al., 2017; Wang et al., 2017a;b; Park et al., 2018; Hu et al., 2018; Chen et al., 2019; Cao et al., 2019; Bello et al., 2019) of global feature interaction methods, shown in Figure 1b, only modifies intermediate features, namely, inputs of neurons, which corresponds to ”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, methodology): “based global reasoning unit [13] before the final classifier.”
Bidirectional Graph Reasoning Network for Panoptic Segmentation (2020, methodology): “Moreover, different from [4, 17, 32] that use a single graph for reasoning, our method aims to build a Graph Connection Module, whose nodes have strong semantics (rather than ambiguous nodes in [4]) and are hence more explainable and capable of encoding various relations.”
Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation (2021, background): “Deep learning-based approaches have achieved great success in the semantic segmentation [24, 43, 2, 7, 42, 3, 17, 10], thanks to a large amount of fully annotated data.”