Conditional Convolutions for Instance Segmentation

By Zhi Tian, Chunhua Shen, Hao Chen. 2020

749 citations recorded by the source corpus.

We propose a simple yet effective instance segmentation framework, termed CondInst (conditional convolutions for instance segmentation). Top-performing instance segmentation methods such as Mask R-CNN rely on ROI operations (typically ROIPool or ROIAlign) to obtain the final instance masks. In contrast, we propose to solve instance segmentation from a new perspective. Instead of using instance-wise ROIs as inputs to a network of fixed weights, we employ dynamic instance-aware networks, conditioned on instances. CondInst enjoys two advantages: 1) Instance segmentation is solved by a fully convolutional network, eliminating the need for ROI cropping and feature alignment. 2) Due to the much improved capacity of dynamically-generated conditional convolutions, the mask head can be very compact (e.g., 3 conv. layers, each having only 8 channels), leading to significantly faster inference. We demonstrate a simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed. On the COCO dataset, we outperform a few recent methods including well-tuned Mask RCNN baselines, without longer training schedules needed. Code is available: this https URL

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Masked-attention Mask Transformer for Universal Image Segmentation (2021, methodology): “, using dynamic kernels [3, 49, 56] or clustering algorithms [11, 29].”
Per-Pixel Classification is Not All You Need for Semantic Segmentation (2021, methodology): “The recently proposed Max-DeepLab [38] removes the dependence on box predictions for panoptic segmentation with conditional convolutions [35, 40].”
K-Net: Towards Unified Image Segmentation (2021, methodology): “These methods all rely on NMS to eliminate duplicated instance masks, which hinders end-to-end training.”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Our mask prediction head is inspired by CondInst [85] and SOLOv2 [92], which extend dynamic convolution [43, 99] to instance segmentation.”
End-to-End Video Instance Segmentation with Transformers (2020, background): “While significant progress has been witnessed in instance segmentation of images [5,9,22,25–27], much less effort was spent on segmenting instances in videos.”
Conditional DETR for Fast Training Convergence (2021, methodology): “Dynamic filter network [16] learns the convolutional kernels from the input, which is applied to instance segmentation in CondInst [38] and SOLOv2 [42] for learning instance-dependent convolutional kernels.”
Box-supervised Instance Segmentation with Level Set Evolution (2022, methodology): “BoxInst [44] uses the color-pairwise affinity with box constraint under an efficient RoI-free CondInst framework [43].”
SOLOv2: Dynamic and Fast Instance Segmentation (2020, methodology): “Note that the concurrent work in [30] also applies dynamic convolutions for instance segmentation by extending the framework of BlendMask [4].”
SOLQ: Segmenting Objects by Learning Queries (2021, methodology): “For example, CondInst [9] based on FCOS [12] employs dynamic convolutions [14] to perform instance segmentation.”
Crossover Learning for Fast Online Video Instance Segmentation (2021, methodology): “Ldet and Lseg denote the object detection loss and stillimage instance segmentation loss in [52].”
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation (2022, methodology): “The object queries, updated by multiple transformer decoders, are employed as dynamic convolution weights (with kernel size 1 × 1) [41, 78, 86] to obtain the prediction Z ∈ RHW×N that consists of N binary masks.”
BoxInst: High-Performance Instance Segmentation with Box Annotations (2020, methodology): “For example, built on the detector FCOS [29], CondInst [28] only adds very compact dynamic mask heads to predict instance masks, and thus only introduces less than 10% computation time, compared to FCOS.”
ResT: An Efficient Transformer for Visual Recognition (2021, background): “Deep learning backbone architectures have been evolved for years and boost the performance of computer vision tasks such as classiﬁcation [5, 25, 32, 10], object detection [2, 39, 17, 24], and instance segmentation [9, 23, 30], etc.”
Instances as Queries (2021, positive): “Queries), a query based end-to-end instance segmentation method driven by parallel supervision on dynamic mask heads [26, 46, 44].”
Transformer-Based Visual Segmentation: A Survey (2023, background): “The designs of mask heads are various, including FCN heads [76], [80], diverse mask encodings [81], and dynamic kernels [77], [82].”
Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss (2021, background): “, 2020b) and CondInst (Tian et al., 2020), which assign different instances to different channels.”
Pointly-Supervised Instance Segmentation (2021, methodology): “For BoxInst [52] we use performance reported by its authors.”
Mask3D: Mask Transformer for 3D Semantic Instance Segmentation (2022, background): “convolutions [25, 54] which is similar in spirit to our mask prediction mechanism.”