Path Aggregation Network for Instance Segmentation

By Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, Jiaya Jia. 2018

7,881 citations recorded by the source corpus.

The way that information propagates in neural networks is of great importance. In this paper, we propose Path Aggregation Network (PANet) aiming at boosting information flow in proposal-based instance segmentation framework. Specifically, we enhance the entire feature hierarchy with accurate localization signals in lower layers by bottom-up path augmentation, which shortens the information path between lower layers and topmost feature. We present adaptive feature pooling, which links feature grid and all feature levels to make useful information in each level propagate directly to following proposal subnetworks. A complementary branch capturing different views for each proposal is created to further improve mask prediction. These improvements are simple to implement, with subtle extra computational overhead. Yet they are useful and make our PANet reach the 1st place in the COCO 2017 Challenge Instance Segmentation task and the 2nd place in Object Detection task without large-batch training. PANet is also state-of-the-art on MVD and Cityscapes.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Rethinking ImageNet Pre-Training (2018, methodology): “Learning curves of AP on COCO val2017 using Mask R-CNN with R50-FPN and SyncBN [34, 27] (that synchronizes batch statistics across GPUs).”
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, neutral): “0 PANet [62] COCO - 36.”
Objects as Points (2019, background): “Sophisticated two-stage detectors [31,35,48,63] are more accurate, but also slower.”
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression (2019, background): “The dominant trend for improving performance of applications utilizing deep neural networks is to propose either a better architecture backbone [15, 13] or a better strategy to extract reliable local features [6].”
UPSNet: A Unified Panoptic Segmentation Network (2019, background): “Based on common practice in semantic [4, 40] and instance segmentation [29, 24], we expect the performance to be further improved with BN layers fine-tuned.”
Panoptic Feature Pyramid Networks (2019, background): “All recent winners of the COCO detection challenges have built on Mask R-CNN [24] with FPN [36], including in 2017 [39, 45] and 2018.”
3D Instance Segmentation of MVS Buildings (2021, methodology): “PANet uses a bottomup annotation structure to shorten the information path and enhances the feature pyramid with accurate localization signals existing at low levels [20].”
Towards Real-World Prohibited Item Detection: A Large-Scale X-ray Benchmark (2021, background): “As many previous works [18, 22] show the importance of multi-scale feature fusion, we think it is the key technology to solve the problem of prohibited item detection.”
Hybrid Task Cascade for Instance Segmentation (2019, methodology): “Synchronized Batch Normalization [34, 28] is used in the backbone and heads.”
Deformable DETR: Deformable Transformers for End-to-End Object Detection (2020, background): “Most works (Liu et al., 2018a; Parmar et al., 2018; Child et al., 2019; Huang et al., 2019; Ho et al., 2019; Hu et al., 2019; Parmar et al., 2019; Qiu et al., 2019; Beltagy et al., 2020; Ainslie et al., 2020; Zaheer et al., 2020) follow this paradigm.”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “This stacked decoder is similar to feature pyramid networks [58, 63, 84, 76] designed for pyramidal anchor predictions [64], but our purpose here is only to aggregate multi-scale features, i.”
EfficientDet: Scalable and Efficient Object Detection (2019, methodology): “…to output node if they are at the same level, in order to fuse more features without adding much cost, as shown in Figure 2(f); Third, unlike PANet [19] that only has one top-down and one bottom-up path, we treat each bidirectional (top-down & bottom-up) path as one feature network layer, and…”
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection (2019, background): “Recently, PANet [25] shows adding an extra bottomup pathway on FPN features improves feature representations for lower resolution features.”
MONet: Unsupervised Scene Decomposition and Representation (2019, background): “While MONet is partly about instance segmentation, its recurrent decomposition process bears little similarity to state-of-the-art instance segmentation models like Mask R-CNN [He et al., 2017] and PANet [Liu et al., 2018].”
Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, methodology): “Based on Mask R-CNN, PANet (Liu et al. 2018) introduces bottom-up path augmentation, adaptive feature pooling, and fully-connected fusion to enhance the feature representation to improve the accuracy of instance segmentation.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “Table 11 reports the comparison of our network to state-of-the-art single-model object detectors on COCO test-dev without using multi-scale training and multi-scale testing that are done in [85], [97], [110], [115], [127], [128].”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “[10] Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia.”
Context Encoding for Semantic Segmentation (2018, methodology): “Concurrent work also implement SyncBN by first calculating the global mean and then the variance, which requires synchronizing twice in each iteration [34, 43].”