Exploiting Cyclic Symmetry in Convolutional Neural Networks

By Sander Dieleman, Jeffrey De Fauw, Koray Kavukcuoglu. 2016

381 citations recorded by the source corpus.

Many classes of images exhibit rotational symmetry. Convolutional neural networks are sometimes trained using data augmentation to exploit this, but they are still required to learn the rotation equivariance properties from the data. Encoding these properties into the network architecture, as we are already used to doing for translation equivariance by using convolutional layers, could result in a more efficient use of the parameter budget by relieving the model from learning them. We introduce four operations which can be inserted into neural network models as layers, and which can be combined to make these models partially equivariant to rotations. They also enable parameter sharing across different orientations. We evaluate the effect of these architectural modifications on three datasets which exhibit rotational symmetry and demonstrate improved performance with smaller models.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Group Normalization (2018, background): “However, in addition to orientations (SIFT [14], HOG [15], or [44,45]), there are many factors that could lead to grouping, e.”
Deformable Convolutional Networks (2017, background): “Some works are devoted for speciﬁc transformations such as symmetry [13, 9], scale [29], and rotation [53].”
Why do deep convolutional networks generalize so poorly to small image transformations? (2018, methodology): “Alternatively, one could use specially designed features such as SIFT in which invariance is hard coded [7, 29, 30, 31] or neural network architectures that explicitly enforce invariance [36, 19, 5, 6, 11, 12, 47, 46, 8].”
Stacked Capsule Autoencoders (2019, background): “Geometrical Reasoning Other attempts at incorporating geometrical knowledge into neural networks include exploiting equivariance properties of group transformations (Cohen and Welling, 2016) or new types of convolutional ﬁlters (Oyallon and Mallat, 2015; Dieleman et al., 2016).”
Group Equivariant Stand-Alone Self-Attention For Vision (2020, background): “The translation equivariance of CNNs has been extended to additional symmetries ranging from planar rotations (Dieleman et al., 2016; Marcos et al., 2017; Worrall et al., 2017; Weiler et al., 2018b; Li et al., 2018; Cheng et al., 2018; Hoogeboom et al., 2018; Bekkers et al., 2018; Veeling et al.,…”
Set2Graph: Learning Graphs From Sets (2020, background): “…Xu et al., 2019; Kondor et al., 2018; Maron et al., 2019b;a), learning images with rotational and reﬂectional symmetries (Cohen & Welling, 2016a;b; Dieleman et al., 2016; Worrall et al., 2017), learning functions on spheres (Cohen et al., 2018; Esteves et al., 2017) and learning general 3D…”
General E(2)-Equivariant Steerable CNNs (2019, methodology): “As a result, we are able to implement a wide range of equivariant models, covering regular GCNNs [6, 7, 8, 9, 10, 11], classical Steerable CNNs [1], Harmonic Networks [12], gated Harmonic Networks [2], Vector Field Networks [13], Scattering Transforms [14, 15, 16, 17, 18] and entirely new…”
Steerable CNNs (2016, background): “Steerable filters were first studied for applications in signal processing and low-level vision (Freeman & Adelson, 1991; Greenspan et al., 1994; Simoncelli & Freeman, 1995). More or less explicit connections between steerability and group representation theory have been observed by Lenz (1989); Koe”
Group Equivariant Convolutional Networks (2016, background): “A networkΦ can be non-injective, meaning that non-identical vectorsx and y in the input space become identical in the output space (for example, two instances of a face may be mapped onto a single vector indicating the presence of any face).”
Equivariance Through Parameter-Sharing (2017, methodology): “Going beyond augmentation, several methods directly apply the group-action, in one way or another, by transforming the data or its encodings using group members (Jaderberg et al., 2015; Anselmi et al., 2013; Dieleman et al., 2016).”
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations (2017, background): “…knowledge about the nature of input transformations (e.g., rotations, translations, scaling for images, etc.) (Schmidt and Roth, 2012; Bruna and Mallat, 2013; Anselmi et al., 2014, 2016; Cohen and Welling, 2016; Dieleman et al., 2016; Haasdonk et al., 2005; Mroueh et al., 2015; Raj et al., 2017).”
Learning Steerable Filters for Rotation Equivariant CNNs (2017, background): “The work [19] introduces four operations which are easily included into existing networks and enrich both the batch- and feature dimension with transformed versions of their content.”
PointCNN: Convolution On X-Transformed Points (2018, background): “While these symmetric pooling based approaches, as well as those in [10, 58, 36], have guarantee in achieving order invariance, they come with a price of throwing away information.”
CubeNet: Equivariance to 3D Rotation and Translation (2018, background): “Recently there has been an explosion of interest into CNNs with predefined transformation equivariances, beyond translation [8, 50, 11, 29, 16, 42, 36, 14, 19, 31, 18, 22, 15, 55, 49, 26, 25, 48, 33, 28, 9].”
3D G-CNNs for Pulmonary Nodule Detection (2018, background): “This is a burgeoning field of research, with G-CNNs being developed for discrete 2D rotation and reflection symmetries (15; 19; 16), continuous planar rotations (20; 21; 22), 3D rotations of spherical signals (23), and permutations of nodes in a graph (24).”