Active Convolution: Learning the Shape of Convolution for Image Classification

By Yunho Jeon, Junmo Kim. 2017

183 citations recorded by the source corpus.

In recent years, deep learning has achieved great success in many computer vision applications. Convolutional neural networks (CNNs) have lately emerged as a major approach to image classification. Most research on CNNs thus far has focused on developing architectures such as the Inception and residual networks. The convolution layer is the core of the CNN, but few studies have addressed the convolution unit itself. In this paper, we introduce a convolution unit called the active convolution unit (ACU). A new convolution has no fixed shape, because of which we can define any form of convolution. Its shape can be learned through backpropagation during training. Our proposed unit has a few advantages. First, the ACU is a generalization of convolution, it can define not only all conventional convolutions, but also convolutions with fractional pixel coordinates. We can freely change the shape of the convolution, which provides greater freedom to form CNN structures. Second, the shape of the convolution is learned while training and there is no need to tune it by hand. Third, the ACU can learn better than a conventional unit, where we obtained the improvement simply by changing the conventional convolution to an ACU. We tested our proposed method on plain and residual networks, and the results showed significant improvement using our method on various datasets and architectures in comparison with the baseline. Code is available at https://github.com/jyh2986/Active-Convolution.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Local Relation Networks for Image Recognition (2019, methodology): “There are also some methods that learn a meta-network {θ} on input features to generate adaptive aggregation weights [15] or an adaptive aggregation scope across spatial positions [6], or learn a fixed prior about spatial aggregation scope (Ω) [14].”
Deformable ConvNets V2: More Deformable, Better Results (2018, background): “In active convolution [25], which is contemporary with Deformable ConvNets, convolutional kernel offsets are learned via back-propagation.”
Generative Image Inpainting with Contextual Attention (2018, background): “[18] propose to learn spatially attentive or active convolutional kernels.”
Dynamic Refinement Network for Oriented and Densely Packed Object Detection (2020, background): “Active convolution [14] augments the sampling locations in the convolutional layers with offsets.”
Learning Spherical Convolution for Fast Features from 360° Imagery (2017, background): “Active convolution [22] learns the kernel shape together with the weights for a more general receptive field, and deformable convolution [7] goes one step further by predicting the receptive field location.”
Interpretable R-CNN (2017, unknown): “Spatial attention-like mechanism has been widely studied in deep neural network based systems, including, but not limited to, the seminal spatial transform network [30] which warps the feature map via a global parametric transformation such as affine transformation, the exploration of global average”
Adaptive Convolutional Filter Generation for Natural Language Understanding (2017, background): “However, their model needs considerably more parameters to achieve performance gains over traditional CNNs. Johnson and Zhang (2015) utilized region embeddings as input to improve text classification accuracy, which can be considered as augmenting phrase-specific features.”
Interaction-And-Aggregation Network for Person Re-Identification (2019, background): “The works [19, 9] augmented the sampling locations in the convolution with offsets and learn the offsets via back-propagation end-to-end.”
Funnel Activation for Visual Recognition (2020, methodology): “Another type of methods learn the spatial dependency adaptively, such as STN [22], active convolution [24], deformable convolution [7].”
Dynamic Region-Aware Convolution (2020, background): “[18] introduces a convolution unit called active convolution unit (ACU), which produces unfixed shape because they can learn to any form through back propagation during training.”
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model (2017, methodology): “Recently, Jeon & Kim [14] has used similar ideas to extend the convolution operations in CNN.”
Learning Context-Sensitive Convolutional Filters for Text Processing (2017, background): “Our work is in line with previous efforts on improving the adaptivity and ﬂexibility of convolutional neural networks (Jeon and Kim, 2017; De Brabandere et al., 2016).”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “More recent research focuses on learning the shape of convolutional kernels [172], [173].”
Geometric Structure Based and Regularized Depth Estimation From 360 Indoor Imagery (2020, background): “Moreover, in [8], Deng et al. apply a deformable convolution [4] and an active convolution [16] to ﬁsheye lenses, another distortion challenge.”
Kernel Transformer Networks for Compact Spherical Convolution (2018, methodology): “Active convolution [21] and deformable convolution [13] model geometric transformations using the receptive ﬁeld of the kernel.”
Attentive Recurrent Neural Network for Weak-supervised Multi-label Image Classification (2018, background): “To solve multi-label image classification, researchers [13, 18, 20, 21, 48] try to overcome two main challenges, (1) how to learn the discriminative feature representations and (2) how to effectively model the complex label relations.”
Towards Interpretable Object Detection by Unfolding Latent Structures (2019, background): “…class speciﬁc activation maps for weakly-supervised discriminative localization [68], the deformable convolution network [9] and active convolution [28], and more explicit attention based work in image caption and visual question answering (VQA) such as the show-attend-tell work [63] and the…”