How Much Position Information Do Convolutional Neural Networks Encode?

By Md. Amirul Islam, Sen Jia, Neil D. B. Bruce. 2020

408 citations recorded by the source corpus.

In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An implication of this is that a filter may know what it is looking at, but not where it is positioned in the image. Information concerning absolute position is inherently useful, and it is reasonable to assume that deep CNNs may implicitly learn to encode this information if there is a means to do so. In this paper, we test this hypothesis revealing the surprising degree of absolute position information that is encoded in commonly used neural networks. A comprehensive set of experiments show the validity of this hypothesis and shed light on how and where this information is represented while offering clues to where positional information is derived from in deep CNNs.

Subfields: Computer Vision and Pattern Recognition, Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

In Safety, Robustness & Interpretability

Citations in the corpus

Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs (2021, neutral): “Recent works [14, 18, 1, 16] showed that CNNs exploit absolute position information.”
PVT v2: Improved baselines with Pyramid Vision Transformer (2021, methodology): “[17, 19, 20], we remove the fixed-size position encoding [1], and introduce zero padding position encoding into PVT.”
On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location (2020, neutral): “[45].”
EMCAD: Efficient Multi-Scale Convolutional Attention Decoding for Medical Image Segmentation (2024, background): “While the SA excels at capturing global information, it is less adept at understanding the local spatial context [13, 28].”
StyleSwin: Transformer-based GAN for High-resolution Image Generation (2021, methodology): “However, when substituting the convolutional layers with transformers that use RPE, one thing is rarely noticed: ConvNets could infer the absolute positions by leveraging the clues from the zero paddings [22,31] yet such feature is missing in Swin blocks using RPE.”
CMT: Convolutional Neural Networks Meet Vision Transformers (2021, background): “Besides, vision transformers ignore the local relation [38] and the structure information [26] inside the patch.”
What shapes feature representations? Exploring datasets, architectures, and training (2020, background): “Several papers have [14, 16] shown that it is possible to decode “categoryorthogonal” features from CNNs trained on naturalistic datasets.”
SOLOv2: Dynamic and Fast Instance Segmentation (2020, background): “The results are reasonably good because that CNNs can implicitly learn the absolute position information from the commonly used zero-padding operation, as revealed in [15].”
Graph Neural Networks with Learnable Structural and Positional Representations (2021, background): “For instances, ConvNets intrinsically consider regular spatial structure for the position of pixels (Islam et al., 2020), RNNs also build on the sequential structure of the word positions, and Transformers employ positional encoding of words (see Dufter et al. (2021) for a review).”
High-fidelity GAN Inversion with Padding Space (2022, background): “It is revealed that padding would implicitly offer the absolute spatial location as the inductive bias [19,20,27], which may bring blind spots for object detectors [4].”
Medical Image Segmentation via Cascaded Attention Decoding (2023, background): “However, the selfattention used in transformers limits their ability to learn local (contextual) relations among pixels [7, 16].”
TS2Vec: Towards Universal Representation of Time Series (2021, background): “It has been proved that convolutional layers can implicitly encode positional information into into latent space (Islam, Jia, and Bruce 2019; Kayhan and Gemert 2020).”
TransNeXt: Robust Foveal Visual Perception for Vision Transformers (2023, positive): “In our evaluation of UperNet with multi-scale and flip augmentations (Table 14), we present test results under both interpolation and extrapolation for a balanced comparison, highlighting the influence of different schemes on multi-scale performance.”
TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication (2021, background): “In [36] it is shown that the adding of zero padding can provide an absolute position information to convolution.”
Vision Transformers for Single Image Dehazing (2022, methodology): “In contrast to CSwin Transformer, we use reflection padding instead of zero padding because we do not need it to encode position information implicitly [71].”