Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data

By David W. Romero, Mark Hoogendoorn. 2019

25 citations recorded by the source corpus.

Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping. Even though some combinations of transformations might never appear (e.g. an upright face with a horizontal nose), current equivariant architectures consider the set of all possible transformations in a transformation group when learning feature representations. Contrarily, the human visual system is able to attend to the set of relevant transformations occurring in the environment and utilizes this information to assist and improve object recognition. Based on this observation, we modify conventional equivariant feature mappings such that they are able to attend to the set of co-occurring transformations in data and generalize this notion to act on groups consisting of multiple symmetries. We show that our proposed co-attentive equivariant neural networks consistently outperform conventional rotation equivariant and rotation & reflection equivariant neural networks on rotated MNIST and CIFAR-10.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Group Equivariant Stand-Alone Self-Attention For Vision (2020, background): “More recently, attention has been introduced to group equivariant architectures (Romero & Hoogendoorn, 2019; Diaconu & Worrall, 2019; Romero et al., 2020a).”
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, methodology): “Furthermore, we would like to replace group convolutions by their attentive counterparts [55, 56] to construct spectrotemporal attention maps indicating the importance of particular time-frequency atoms for a given task.”
Attentive Group Equivariant Convolutional Networks (2020, methodology): “For vision tasks, self-attention has been proposed to encode visual co-occurrences in data (Hu et al., 2018; Wang et al., 2018; Park et al., 2018; Woo et al., 2018; Cao et al., 2019; Bello et al., 2019; Ramachandran et al., 2019; Romero & Hoogendoorn, 2020).”