Rotation Equivariant Vector Field Networks

By Diego Marcos, Michele Volpi, Nikos Komodakis, Devis Tuia. 2016

297 citations recorded by the source corpus.

In many computer vision tasks, we expect a particular behavior of the output with respect to rotations of the input image. If this relationship is explicitly encoded, instead of treated as any other variation, the complexity of the problem is decreased, leading to a reduction in the size of the required model. In this paper, we propose the Rotation Equivariant Vector Field Networks (RotEqNet), a Convolutional Neural Network (CNN) architecture encoding rotation equivariance, invariance and covariance. Each convolutional filter is applied at multiple orientations and returns a vector field representing magnitude and angle of the highest scoring orientation at every spatial location. We develop a modified convolution operator relying on this representation to obtain deep architectures. We test RotEqNet on several problems requiring different responses with respect to the inputs’ rotation: image classification, biomedical image segmentation, orientation estimation and patch matching. In all cases, we show that RotEqNet offers extremely compact models in terms of number of parameters and provides results in line to those of networks orders of magnitude larger.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Group Equivariant Stand-Alone Self-Attention For Vision (2020, background): “The translation equivariance of CNNs has been extended to additional symmetries ranging from planar rotations (Dieleman et al., 2016; Marcos et al., 2017; Worrall et al., 2017; Weiler et al., 2018b; Li et al., 2018; Cheng et al., 2018; Hoogeboom et al., 2018; Bekkers et al., 2018; Veeling et al., 20”
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (2020, methodology): “This approach is easy to implement, and has also been used when the feature maps are vector ﬁelds (Zhou et al., 2017; Marcos et al., 2017), and with other representations (Cohen and Welling, 2016b), but only on image data where locations are discrete and the group cardinality is small.”
General E(2)-Equivariant Steerable CNNs (2019, methodology): “The original paper [13] used a different convention arg max : R → {0, .”
Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds (2018, background): “Other authors have investigated the problems of rotation equivariance in 2D, such as Zhou et al. [4], Gonzalez et al. [5], Li et al. [6].”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, background): “…Hy et al., 2018; Maron et al., 2019), translations and rotations of the plane and 3D space (Oyallon & Mallat, 2015; Cohen & Welling, 2016; 2017; Marcos et al., 2017; Weiler et al., 2018b;a; Worrall et al., 2017; Worrall & Brostow, 2018; Winkels & Cohen, 2018; Winkens et al., 2018; Thomas et…”
On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups (2018, background): “The recent papers (Marcos et al., 2017) and (Worrall et al., 2017) extend these architectures by considering continuous rotations at each point of the visual ﬁeld.”
Learning SO(3) Equivariant Representations with Spherical CNNs (2017, methodology): “[18] use a rotational ﬁlter orbit to produce oriented feature maps and rotationally invariant features, and Lenc and Vedaldi [19] propose a transformation layer which acts as a group-convolution by ﬁrst permuting then transforming by a linear ﬁlter.”
Learning Steerable Filters for Rotation Equivariant CNNs (2017, background): “These ideas were extended to networks which additionally propagate the orientation of the maximum response [22].”
CubeNet: Equivariance to 3D Rotation and Translation (2018, background): “Recently there has been an explosion of interest into CNNs with predefined transformation equivariances, beyond translation [8, 50, 11, 29, 16, 42, 36, 14, 19, 31, 18, 22, 15, 55, 49, 26, 25, 48, 33, 28, 9].”
Deep Rotation Equivariant Network (2017, background): “Recently, the vector field network proposed by [18] applies interpolation to deal with rotation of general degrees.”
Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images (2020, methodology): “Instead of carrying forward all orientations throughout the network, VF-CNNs collapse the orientation dependent feature maps to two feature maps, representing magnitude and angle.”
Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models (2018, methodology): “As shown in Marcos et al. (2016), it can be applied to various applications requiring rotation invariance, equivariance or even covariance, which opens doors for the application of RotEqNet to tackle problems of detection (cars, airplanes, trees) or regression (superresolution, biophysical…”
SE(3)-equivariant prediction of molecular wavefunctions and electronic densities (2021, background): “reflecting rotational equivariance [38], energy conservation, physical laws or symmetries (e.”
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness (2019, methodology): “the mathematical model for rotations and translations has led to carefully hand-engineered architectures that incorporate spatial invariance directly [19, 24, 8, 27, 44, 42, 12, 40].”
Scale equivariance in CNNs with vector fields (2018, methodology): “In this contribution, we propose to use a vector ﬁeld representation similar to the one in (Marcos et al., 2017) to obtain a locally scale-equivariant deep CNN.”
Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data (2019, background): “…by stacking convolutional feature maps, several approaches have emerged to extend equivariance to, e.g., planar rotations (Dieleman et al., 2016; Marcos et al., 2017; Weiler et al., 2018; Li et al., 2018), spherical rotations (Cohen et al., 2018; Worrall & Brostow, 2018; Cohen et al., 2019),…”