TI-POOLING: Transformation-Invariant Pooling for Feature Learning in Convolutional Neural Networks

By Dmitry Laptev, Nikolay Savinov, Joachim M. Buhmann, Marc Pollefeys. 2016

273 citations recorded by the source corpus.

In this paper we present a deep neural network topology that incorporates a simple to implement transformationinvariant pooling operator (TI-POOLING). This operator is able to efficiently handle prior knowledge on nuisance variations in the data, such as rotation or scale changes. Most current methods usually make use of dataset augmentation to address this issue, but this requires larger number of model parameters and more training data, and results in significantly increased training time and larger chance of under-or overfitting. The main reason for these drawbacks is that that the learned model needs to capture adequate features for all the possible transformations of the input. On the other hand, we formulate features in convolutional neural networks to be transformation-invariant. We achieve that using parallel siamese architectures for the considered transformation set and applying the TI-POOLING operator on their outputs before the fully-connected layers. We show that this topology internally finds the most optimal "canonical" instance of the input image for training and therefore limits the redundancy in learned features. This more efficient use of training data results in better performance on popular benchmark datasets with smaller number of parameters when comparing to standard convolutional neural networks with dataset augmentation and to other baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

General E(2)-Equivariant Steerable CNNs (2019, neutral): “69 [40] - 1.”
RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks (2018, methodology): “While deep convolutional neural networks (CNN) have been widely used in computer vision and image processing applications, they are not designed to handle large group actions like rotations, which degrade the performance of CNN in many tasks [2, 9, 15, 21, 23].”
Harmonic Networks: Deep Translation and Rotation Equivariance (2016, background): “[17] do the same for a broader class of global image transformations, and propose a novel per-pixel pooling technique for output fusion.”
Understanding Image Representations by Measuring Their Equivariance and Equivalence (2014, methodology): “This can be as simple as averaging the features (Anselmi et al. 2016), max-pooling (Laptev et al. 2016; Cohen and Welling 2016) or simply by exploiting the group symmetry [such as ignoring the gradient ‘sign’ in Dalal and Triggs (2005) for vertical flip invariance].”
Learning Steerable Filters for Rotation Equivariant CNNs (2017, background): “In [21], several rotated versions of the same image are sent through a conventional CNN.”
CubeNet: Equivariance to 3D Rotation and Translation (2018, background): “Fasel & Gatica-Perez [7], Laptev et al.[12], and Gonzalez et al.[11] average classifier predictions on multiple rotated copies of an input.”
Oriented Response Networks (2017, methodology): “In Deep Convolution Neural Networks (DCNNs), the inherent properties of convolution and pooling alleviate the effect of local transitions and warps; however, lacking the capability to handle large image rotation limits DCNN’s performance in many visual tasks including object boundary detection [16, ”
Deep Rotation Equivariant Network (2017, methodology): “In other words, this means that all of feature maps would be convolved by rotated copies of a single ﬁlter, which is utilized by [1], [14], [20].”
Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models (2017, methodology): “To address this, we implemented a simple modification (in the spirit of the TI-Pooling [14]) that applies Kd-network (depth 10) to 20 different random rotations of a model and performs max-pooling over the produced representations followed by three fully connected layers to produce final shape descr”
Scale equivariance in CNNs with vector fields (2018, methodology): “In the example above, the layer will detect both numbers in the image (scale invariance) and will also provide the information at which scale the ﬁves were detected, allowing for scale equivariance. of (Laptev et al., 2016), enforcing scale invariance can lead to a loss in performance.”
Robustness of Rotation-Equivariant Networks to Adversarial Perturbations (2018, background): “…the existing approaches fall into two categories: the ones based on the rotation of the input images (Jaderberg et al., 2015; Dieleman et al., 2016; Laptev et al., 2016; Henriques & Vedaldi, 2017; Esteves et al., 2018), and the ones based on constraints on the structure of the ﬁlters (Cohen &…”
Sparsity Invariant CNNs (2017, background): “The most commonly used solution to ensure robustness is data augmentation [35, 36, 59].”
Beyond Planar Symmetry: Modeling Human Perception of Reflection and Rotation Symmetries in the Wild (2017, background): “…in the deep learning/CNN community where researchers are seeking networks that are rotation/reﬂection or afﬁne invariant to input images [11, 18, 31, 77], our work explicitly acknowledges (near) reﬂection and rotation symmetries in the raw data regardless of the transformations applied on the…”
Self-Supervised Representation Learning by Rotation Feature Decoupling (2019, background): “[26] extract max-pooled activation of multiple rotational copies of images.”
Matrix capsules with EM routing (2018, methodology): “This approach is more parameter-efﬁcient than data augmentation approaches, duplicating feature maps, or duplicating ﬁlters (Fasel & Gatica-Perez (2006), Laptev et al. (2016)).”