Mind the Pad - CNNs can Develop Blind Spots

By Bilal Alsallakh, Narine Kokhlikyan, Vivek Miglani, Jun Yuan, Orion Reblitz-Richardson. 2020

85 citations recorded by the source corpus.

We show how feature maps in convolutional networks are susceptible to spatial bias. Due to a combination of architectural choices, the activation at certain locations is systematically elevated or weakened. The major source of this bias is the padding mechanism. Depending on several aspects of convolution arithmetic, this mechanism can apply the padding unevenly, leading to asymmetries in the learned weights. We demonstrate how such bias can be detrimental to certain tasks such as small object detection: the activation is suppressed if the stimulus lies in the impacted area, leading to blind spots and misdetection. We propose solutions to mitigate spatial bias and demonstrate how they can improve model accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

High-fidelity GAN Inversion with Padding Space (2022, background): “side effect of constant padding used in CNNs has recently been studied [3, 18, 19, 23, 26, 45].”
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer (2022, background): “When a short video clip is encoded, the zero paddings in the temporal dimension also get encoded and affect the output tokens, causing an unbalanced effects to tokens at different temporal position [24,33,80,3].”
Simple Baselines for Image Restoration (2022, unknown): “This design choice is inspired by [2].”
Positional Encoding as Spatial Inductive Bias in GANs (2020, background): “While the influence of padding to translation-invariant convolution has been discussed in recent works [2, 18, 23], these studies focus on image classification and detection.”