Vision GNN: An Image is Worth Graph of Nodes

By Kai Han, Yunhe Wang, Jianyuan Guo, Yehui Tang, Enhua Wu. Neural Information Processing Systems, 2022

633 citations recorded by the source corpus.

Network architecture plays a key role in the deep learning-based computer vision system. The widely-used convolutional neural network and transformer treat the image as a grid or sequence structure, which is not flexible to capture irregular and complex objects. In this paper, we propose to represent the image as a graph structure and introduce a new Vision GNN (ViG) architecture to extract graph-level feature for visual tasks. We first split the image to a number of patches which are viewed as nodes, and construct a graph by connecting the nearest neighbors. Based on the graph representation of images, we build our ViG model to transform and exchange information among all the nodes. ViG consists of two basic modules: Grapher module with graph convolution for aggregating and updating graph information, and FFN module with two linear layers for node feature transformation. Both isotropic and pyramid architectures of ViG are built with different model sizes. Extensive experiments on image recognition and object detection tasks demonstrate the superiority of our ViG architecture. We hope this pioneering study of GNN on general visual tasks will provide useful inspiration and experience for future research. The PyTorch code is available at https://github.com/huawei-noah/Efficient-AI-Backbones and the MindSpore code is available at https://gitee.com/mindspore/models.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/270195c8ecc14b98a206461282b88f8fae798932

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xinchao Wang
2. Jiashi Feng
3. Shuicheng Yan
4. Weihao Yu
5. Pan Zhou
6. Dacheng Tao
7. Yichen Zhou
8. Chenyang Si

Citations in the corpus

• MetaFormer Baselines for Vision (2022, methodology): “The power of MetaFormer can also be verified by the recent models adopting MetaFormer as the general architecture but with different attention-based [40, 57, 70], MLP-based [66, 67, 27, 6, 41], convolution-based [15, 19, 79, 56, 77], hybrid [61, 12, 37, 70] or other types of [64, 20] token mixers.”