Scale equivariance in CNNs with vector fields

By Diego Marcos, Benjamin Kellenberger, Sylvain Lobry, Devis Tuia. 2018

59 citations recorded by the source corpus.

We study the effect of injecting local scale equivariance into Convolutional Neural Networks. This is done by applying each convolutional filter at multiple scales. The output is a vector field encoding for the maximally activating scale and the scale itself, which is further processed by the following convolutional layers. This allows all the intermediate representations to be locally scale equivariant. We show that this improves the performance of the model by over 20% in the scale equivariant task of regressing the scaling factor applied to randomly scaled MNIST digits. Furthermore, we find it also useful for scale invariant tasks, such as the actual classification of randomly scaled digits. This highlights the usefulness of allowing for a compact representation that can also learn relationships between different local scales by keeping internal scale equivariance.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Group Equivariant Stand-Alone Self-Attention For Vision (2020, background): “, 2019b), scaling (Marcos et al., 2018; Worrall & Welling, 2019; Sosnovik et al., 2020; Romero et al., 2020b) and more general symmetry groups (Cohen & Welling, 2016; Kondor & Trivedi, 2018; Weiler & Cesa, 2019; Cohen et al.”
Scale-Equivariant Steerable Networks (2019, methodology): “Following Kanazawa et al. (2014); Marcos et al. (2018); Ghosh & Gupta (2019) we conduct experiments on the MNIST-scale dataset.”
B-Spline CNNs on Lie Groups (2019, background): “Scale equivariance is however addressed in several settings (Henriques & Vedaldi, 2017; Esteves et al., 2018b; Marcos et al., 2018; Tai et al., 2019; Worrall & Welling, 2019; Jaderberg et al., 2015), of which (Worrall & Welling, 2019) is most related.”
Deep Scale-spaces: Equivariance Over Scale (2019, unknown): “…convolutions to scale, but they either suffer from breaking translation symmetry [Henriques and Vedaldi, 2017, Esteves et al., 2017], making the assumption that scalings can be modelled in the same way as rotations [Marcos et al., 2018], or ignoring symmetry constraints [Hilbert et al., 2018].”
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, methodology): “[46] Diego Marcos, Benjamin Kellenberger, Sylvain Lobry, and Devis Tuia.”
Attentive Group Equivariant Convolutional Networks (2020, unknown): “…2020), spherical rotations (Cohen et al., 2018; Worrall & Brostow, 2018; Cohen et al., 2019b; Thomas et al., 2018; Kondor & Trivedi, 2018), scaling (Marcos et al., 2018; Worrall & Welling, 2019; Sos-novik et al., 2020) and general symmetry groups (Cohen & Welling, 2016a; Weiler & Cesa, 2019; Cohen…”
On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location (2020, methodology): “…versions of the training data there are methods speciﬁcally designed to learn geometric transformations in an equivariant or invariant representation [7, 53, 60] where examples include rotation [19, 69, 102, 103, 110], scale [68, 92, 99, 104, 106] and other transformations [17, 26, 38, 57, 90].”