Examining the Impact of Blur on Recognition by Convolutional Networks

By Igor Vasiljevic, Ayan Chakrabarti, Gregory Shakhnarovich. 2016

213 citations recorded by the source corpus.

State-of-the-art algorithms for semantic visual tasks---such as image classification and semantic segmentation---are based on the use of convolutional neural networks. These networks are commonly trained, and evaluated, on large annotated datasets of high-quality images that are free of artifacts. In this paper, we investigate the effect of one such artifact that is quite common in natural capture settings---blur. We show that standard pre-trained network models suffer a significant degradation in performance when applied to blurred images. We investigate the extent to which this degradation is due to the mismatch between training and input image statistics. Specifically, we find that fine-tuning a pre-trained model with blurred images added to the training set allows it to regain much of the lost accuracy. By considering different combinations of sharp and blurred images in the training set, we characterize how much degradation is caused by loss of information, and how much by the uncertainty of not knowing the nature and magnitude of blur. We find that by fine-tuning on a diverse mix of blurred images, convolutional neural networks can in fact learn to generate a blur invariant representation in their hidden layers. Broadly, our results provide practitioners with useful insights for developing vision systems that perform reliably on real world images affected by blur.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Natural Adversarial Examples (2019, background): “ith adversarial data. Our sources of distribution shift also tend to be more realistic, heterogeneous, and varied. Obtaining robustness to varied forms of distribution shift is difﬁcult. For example, [41,12] train on various distortions and show that networks tend to memorize distortions and thereby”
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “In an effort to reduce classiﬁer fragility, Vasiljevic et al. (2016) ﬁne-tune on blurred images.”
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty (2019, background): “Mixing augmentations allows us to generate diverse transformations, which are important for inducing robustness, as a common failure mode of deep models in the arena of corruption robustness is the memorization of ﬁxed augmentations (Vasiljevic et al., 2016; Geirhos et al., 2018).”
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming (2019, background): “Vasiljevic et al. [2016] study the vulnerability of a classiﬁer to blurred images and enhance the performance on blurred images by ﬁne-tuning on them.”
Generalisation in humans and deep neural networks (2018, neutral): “These results are in line with previous ﬁndings reporting human-like processing of chromatic information in DNNs [19] noise and blur [13, 14, 59–61].”
A Study of Face Obfuscation in ImageNet (2021, background): “Researchers have studied visual recognition in the presence of various image degradation, including blurring (Vasiljevic et al., 2016), lens distortions (Pei et al., 2018), and low resolution (Ryoo et al., 2016).”
Google's Cloud Vision API is Not Robust to Noise (2017, background): “Several papers have recently showed that the performance of deep convolutional neural networks drops when the model is tested on distorted inputs, such as noisy or blurred images [13]–[15].”
Biologically inspired sleep algorithm for increased generalization and adversarial robustness in deep neural networks (2020, background): “Likewise, ANNs trained on unperturbed images exhibit reduced performance when images in the test set are distorted, for example, through horizontal translations, blurring, or the addition of compression artifacts (Dodge & Karam (2016); Vasiljevic et al. (2016); Zhou et al. (2017)).”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, unknown): “This agrees with CNN robustness literature [21, 55].”
Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations (2018, methodology): “Specifically, we introduce the IMAGETNET-C dataset for input corruption robustness [55] and the ICONS-50 dataset for surface variation robustness.”
Orthogonal Deep Neural Networks (2019, background): “Existing research suggests fragility of deep learning models to corruptions of such kinds [15], and that fine-tuning on specific corruption types would help, but cannot well generalize to other types of corruptions [17], [52].”
Quality-Agnostic Image Recognition via Invertible Decoder (2021, background): “However, training (or ﬁne-tuning) with such diverse characteristics of data may generate the averaged distribution among corruptions [47], such that it results in an under-ﬁtted (or poor) model for a particular type of data, e.g., HQ images [14] or LQ images [44, 51, 10].”
Quality Resilient Deep Neural Networks (2017, methodology): “This has been shown to achieve good performance in the case of noisy [26] or blurry data [24].”
Testing Robustness Against Unforeseen Adversaries (2019, background): ““Common” visual corruptions such as (non-adversarial) fog, blur, or pixelation have emerged as another avenue for measuring the robustness of computer vision models [26, 12, 27].”
Human and DNN Classification Performance on Images With Quality Distortions (2019, methodology): “Vasiljevic et al. (2016) follow this approach to achieve improved performance on blurred images.”