Deformable Convolutional Networks
By Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei. 2017
6,697 citations recorded by the source corpus.
Convolutional neural networks (CNNs) are inherently limited to model geometric transformations due to the fixed geometric structures in their building modules. In this work, we introduce two new modules to enhance the transformation modeling capability of CNNs, namely, deformable convolution and deformable RoI pooling. Both are based on the idea of augmenting the spatial sampling locations in the modules with additional offsets and learning the offsets from the target tasks, without additional supervision. The new modules can readily replace their plain counterparts in existing CNNs and can be easily trained end-to-end by standard back-propagation, giving rise to deformable convolutional networks. Extensive experiments validate the performance of our approach. For the first time, we show that learning dense spatial transformation in deep CNNs is effective for sophisticated vision tasks such as object detection and semantic segmentation. The code is released at https://github.com/msracver/Deformable-ConvNets.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Jitendra Malik
- Andreas Geiger
- Xiaogang Wang
- Luc Van Gool
- Raquel Urtasun
- Philipp Krahenbuhl
- Kristen Grauman
- Ross B. Girshick
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Trevor Darrell
- Alan L. Yuille
- Jitendra Malik
- Yi Yang
Citations in the corpus
- Exploring Self-Attention for Image Recognition (2020, neutral): “A number of methods learned to reweight convolutional activations or offset the taps of convolutional kernels [5, 15, 34, 36, 46], thus retaining the basic principle of convolutional feature construction.”
- Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “Another related method is deformable convolution [24,100,28], where each point attends to a few points dynamically on an image.”
- On the Relationship between Self-Attention and Convolutional Layers (2019, background): “More generally, fully-attentional models seem to learn a generalization of CNNs where the kernel pattern is learned at the same time as the filters—similar to deformable convolutions (Dai et al., 2017; Zampieri, 2019).”
- Local Relation Networks for Image Recognition (2019, methodology): “There are also some methods that learn a meta-network {θ} on input features to generate adaptive aggregation weights [15] or an adaptive aggregation scope across spatial positions [6], or learn a fixed prior about spatial aggregation scope (Ω) [14].”
- CCNet: Criss-Cross Attention for Semantic Segmentation (2018, background): “SAC [40] and Deformable Convolutional Networks [11] improved the standard convolutional operator to handle the deformation and various scale of objects.”
- Relation Networks for Object Detection (2017, methodology): “• DCN [10].”
- Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “We further use the Deformable Convolutional Networks (DCN) [10] to the ResNet and ResNeXt backbones as well as the last layer of detector towers.”
- Cascade R-CNN: High Quality Object Detection and Instance Segmentation (2019, positive): “To more thoroughly test this claim, a three-stage Cascade R-CNN was implemented with three baseline detectors: Faster R-CNN, R-FCN, and FPN+.”
- Objects as Points (2019, methodology): “Nonetheless, over the past five years [19], this idea has achieved good empirical success [12,21,25,26,31,35,47,48,56,62,63].”
- UPSNet: A Unified Panoptic Segmentation Network (2019, methodology): “Our semantic head consists of a deformable convolution [9] based subnetwork which takes the multi-scale feature from FPN as input.”
- Panoptic Feature Pyramid Networks (2019, methodology): “Recent innovations include Cascade R-CNN [7], deformable convolution [15], and sync batch norm [45].”
- Soft-NMS — Improving Object Detection with One Line of Code (2017, methodology): “Therefore, greedy NMS performs For human detection, Dalal and Triggs [3] demonstrated that a greedy NMS algorithm, favourably to these algo-NMS ,”
- MegDet: A Large Mini-Batch Object Detector (2017, background): “Deformable ConvNets [7] uses the learned offsets to convolve different locations of feature maps, and forces the networks to focus on the objects.”
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, background): “In addition to these architectural advances, there has also been much work on improving individual convolution layers, such as depthwise convolution [64] and deformable convolution [17, 75].”
- K-Net: Towards Unified Image Segmentation (2021, methodology): “On the test-dev split, K-Net with ResNet- 101-FPN backbone even obtains better results than that of UPSNet [61], which uses Deformable Convolution Network (DCN) [16].”
- Cascade R-CNN: Delving Into High Quality Object Detection (2017, positive): “0 Deformable R-FCN [5]* Aligned-Inception-ResNet 37.”
- Dynamic Neural Networks: A Survey (2021, methodology): “TABLE 2 Kernel shape adaptation by dynamically sampling feature pixels [110], [111] or convolutional weights [112].”
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, methodology): “Note that we do not employ the multi-grid method [75, 15, 10], which we found does not improve the performance.”
- Rethinking Atrous Convolution for Semantic Image Segmentation (2017, methodology): “Motivated by multigrid methods which employ a hierarchy of grids of different sizes [4, 69, 5, 57] and following [72, 16], we adopt different atrous rates within block4 to block7 in the proposed model.”