Polar Transformer Networks

By Carlos Esteves, Christine Allen-Blanchette, Xiaowei Zhou, Kostas Daniilidis. 2017

200 citations recorded by the source corpus.

Convolutional neural networks (CNNs) are inherently equivariant to translation. Efforts to embed other forms of equivariance have concentrated solely on rotation. We expand the notion of equivariance in CNNs through the Polar Transformer Network (PTN). PTN combines ideas from the Spatial Transformer Network (STN) and canonical coordinate representations. The result is a network invariant to translation and equivariant to both rotation and scale. PTN is trained end-to-end and composed of three distinct stages: a polar origin predictor, the newly introduced polar transformer module and a classifier. PTN achieves state-of-the-art on rotated MNIST and the newly introduced SIM2MNIST dataset, an MNIST variation obtained by adding clutter and perturbing digits with translation, rotation and scaling. The ideas of PTN are extensible to 3D which we demonstrate through the Cylindrical Transformer Network.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Making Convolutional Networks Shift-Invariant Again (2019, background): “Recent work has strived to build in additional types of in-variances, such as rotation, reﬂection, and scaling (Sifre & Mallat, 2013; Bruna & Mallat, 2013; Esteves et al., 2018; Kanazawa et al., 2014; Worrall et al., 2017; Cohen & Welling, 2016).”
Why do deep convolutional networks generalize so poorly to small image transformations? (2018, background): “…invariance, which they called “shiftability” and showed that it can hold for systems with subsampling (this is related to weak translation invariance as deﬁned by (Lenc and Vedaldi, 2015), see also (Esteves et al., 2017; Cohen and Welling, 2014) for related ideas applied to neural networks).”
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (2020, methodology): “…the logarithm of v − 1 j u i has a simple expression highlighting the fact that it is invariant to scale and rotational transformations of the elements, Now writing out our Monte Carlo estimation of the integral: which is a discretization of the log polar convolution from Esteves et al. (2017).”
Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network (2018, background): “Spherical CNNs are not the only context in which the idea of Fourier space neural networks has recently appeared [18, 19, 5, 7].”
CubeNet: Equivariance to 3D Rotation and Translation (2018, background): “Recently there has been an explosion of interest into CNNs with predefined transformation equivariances, beyond translation [2,3,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19] [20,21,22,23].”
Robustness of Rotation-Equivariant Networks to Adversarial Perturbations (2018, background): “…the ones based on the rotation of the input images (Jaderberg et al., 2015; Dieleman et al., 2016; Laptev et al., 2016; Henriques & Vedaldi, 2017; Esteves et al., 2018), and the ones based on constraints on the structure of the ﬁlters (Cohen & Welling, 2016; 2017; Worrall et al., 2017; Zhou et…”
Deep Scale-spaces: Equivariance Over Scale (2019, unknown): “[Henriques and Vedaldi, 2017, Esteves et al., 2017] both perform a log-polar transform of the signal before passing to a standard CNN. Log-polar transforms reparame-terize the input plane into angle and log-distance from a predeﬁned origin.”
Under review as a conference paper at ICLR 2020 completely free-form sampling in image space : Spatial Transformers (2019, background): “However, this usually requires designs specific to certain kinds of deformation, such as shift, rotation, reflection and scaling (Sifre & Mallat, 2013; Bruna & Mallat, 2013; Kanazawa et al., 2016; Cohen & Welling, 2016; Worrall et al., 2017; Esteves et al., 2018).”
Disc-Aware Ensemble Network for Glaucoma Screening From Fundus Image (2018, background): “2) Due to the pixel-wise mapping, the polar transformation is equivariant to the data augmentation on the original fundus image [33].”
Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey (2021, background): “Approaches like the oriented response network [105], polar transformer network [106], and equivariant transformer network [107], which were proposed for object detection from natural scenes, also provided a quantitative or qualitative analysis of rotation-invariant features.”
Under review as a conference paper at ICLR 2020 completely free-form sampling in image space : Spatial Transformers (2019, background): “However, this usually requires designs specific to certain kinds of deformation, such as shift, rotation, reflection and scaling (Sifre & Mallat, 2013; Bruna & Mallat, 2013; Kanazawa et al., 2016; Cohen & Welling, 2016; Worrall et al., 2017; Esteves et al., 2018).”
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness (2019, methodology): “We compare the robust prediction accuracies from networks trained with the regularizers with three specialized architectures, designed to be equivariant against spatial transformations and translations: (a) G-ResNet44 (GRN) [8] using p4m convolutional layers (90 degree rotations, translations and mi”
Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation (2019, background): “However, this usually requires designs speciﬁc to certain kinds of deformation, such as shift, rotation, reﬂection and scaling (Sifre & Mallat, 2013; Bruna & Mallat, 2013; Kanazawa et al., 2016; Cohen & Welling, 2016; Worrall et al., 2017; Esteves et al., 2018).”