Understanding how image quality affects deep neural networks

By Samuel F. Dodge, Lina Karam. 2016

815 citations recorded by the source corpus.

Image quality is an important practical challenge that is often overlooked in the design of machine vision systems. Commonly, machine vision systems are trained and tested on high quality image datasets, yet in practical applications the input images can not be assumed to be of high quality. Recently, deep neural networks have obtained state-of-the-art performance on many machine vision tasks. In this paper we provide an evaluation of 4 state-of-the-art deep neural network models for image classification under quality distortions. We consider five types of quality distortions: blur, noise, contrast, JPEG, and JPEG2000 compression. We show that the existing networks are susceptible to these quality distortions, particularly to blur and noise. These results enable future work in developing deep neural networks that are more invariant to quality distortions.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering, Media Technology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

In Media Technology

In Safety, Robustness & Interpretability

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, background): “Early investigations in face and natural image recognition found that performance degrades by introducing blur, Gaussian noise, occlusion, and compression artifacts, but less by color distortions [12, 35].”
A Study and Comparison of Human and Deep Learning Recognition Performance under Visual Distortions (2017, background): “However, it has recently been shown that deep networks do not achieve good performance on distorted images [5].”
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming (2019, background): “Dodge and Karam [2016] measure the performance of four state-of-the-art image recognition models on out-of-distribution data and show that CNNs are in particular vulnerable to blur and Gaussian noise.”
Effects of Degradations on Deep Neural Network Architectures (2018, methodology): “to include noisy data in the training itself [2].”
How Image Degradations Affect Deep CNN-Based Face Recognition? (2016, background): “Besides face recognition, image quality effects are also studied in different classification applications, such as object classification [8] and hand-written digit recognition [9].”
Generalisation in humans and deep neural networks (2018, background): “There may be many different reasons for the discrepancy between human and DNN generalisation performance that we find: Are there limitations in terms of the currently used network architectures (as hypothesised by [60]), which may be inferior to the human brain’s intricate computations? Is it a prob”
Examining the Impact of Blur on Recognition by Convolutional Networks (2016, methodology): “For completeness, we also report results on blurring with Gaussian kernels of different standard deviations σ considered in [5].”
“Hands On” Visual Recognition for Visually Impaired Users (2017, background): “A very recent work addresses this issue at a more general level (Dodge and Karam 2016), which we believe would be worth further investigation.”
Classification-Driven Dynamic Image Enhancement (2017, background): “Dodge and Karam [9] analyzed how blur, noise, contrast, and compression hamper the performance of ConvNet architectures for image classiﬁcation.”
On classification of distorted images with deep convolutional neural networks (2017, background): “[12] Samuel Dodge and Lina Karam, “Understanding how image quality affects deep neural networks,” arXiv preprint arXiv:1604.04004, 2016.”
KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment (2019, background): “Image Quality Assessment (IQA) plays an important role in a broad range of applications ranging from image compression to machine vision, and more [1], [2], [3], [4].”
Quality Resilient Deep Neural Networks (2017, methodology): “In our experiments, we use the VGG16 model [22], as it was found to be the most resilient to distortions in [3].”
Towards Robust CNN-based Object Detection through Augmentation with Synthetic Rain Variations (2019, methodology): “Instead of using standard noise distributions as proposed in [12]–[14], [16] or simply one specific rain filter as in [17], we use variations of realistic synthetic rain to modify the datasets and evaluate the robustness.”
Google's Cloud Vision API is Not Robust to Noise (2017, background): “Several papers have recently showed that the performance of deep convolutional neural networks drops when the model is tested on distorted inputs, such as noisy or blurred images [13]–[15].”
Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach (2020, background): “IMAGE QUALITY Due to the mobility of vehicles, the images captured by on-board cameras generally suffer from motion blur, noise, and distortion [21], [22].”