Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment

By Sebastian Bosse, Dominique Maniry, K. Müller, Thomas Wiegand, Wojciech Samek. 2016

1,192 citations recorded by the source corpus.

We present a deep neural network-based approach to image quality assessment (IQA). The network is trained end-to-end and comprises ten convolutional layers and five pooling layers for feature extraction, and two fully connected layers for regression, which makes it significantly deeper than related IQA models. Unique features of the proposed architecture are that: 1) with slight adaptations it can be used in a no-reference (NR) as well as in a full-reference (FR) IQA setting and 2) it allows for joint learning of local quality and local weights, i.e., relative importance of local quality to the global quality estimate, in an unified framework. Our approach is purely data-driven and does not rely on hand-crafted features or other types of prior domain knowledge about the human visual system or image statistics. We evaluate the proposed approach on the LIVE, CISQ, and TID2013 databases as well as the LIVE In the wild image quality challenge database and show superior performance to state-of-the-art NR and FR IQA methods. Finally, cross-database evaluation shows a high ability to generalize between different databases, indicating a high robustness of the learned features.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data (2019, background): “In a concurrent development deep learning has revolutionized the ways that information can be extracted from data resources with groundbreaking successes in areas such as computer vision, natural language processing or voice recognition among many others [4][5][6][7][8][9].”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, background): “The advent of GPU accelerated training [87] and the availability of performant deep learning frameworks [88–92], as well as large datasets [33, 93–97] lead to unprecedented accuracy in a variety of tasks [5, 7, 78, 80, 98, 99].”
End-to-End Blind Image Quality Assessment Using Deep Neural Networks (2018, methodology): “gMAD competition results between MEON and deepIQA [17].”
A Review on State-of-the-Art Power Line Inspection Techniques (2020, background): “Up to now, a DCNN-based detection scheme has been a research hotspots, which has been applied to much research fields, such as human detection [124], autonomous vehicles [125], quality evaluation [126], and industrial detection [127].”
Exploring CLIP for Assessing the Look and Feel of Images (2022, methodology): “Unlike existing NR-IQA methods ( e.g. , MUSIQ) that train different models for different domains ( e.g. , technical quality, aesthet-3 ics quality), CLIP-IQA + needs to store only two prompts for each domain, signiﬁcantly reducing the storage cost.”
Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network (2019, methodology): “To be specific, DB-CNN first plays the role of attacker while deepIQA [26] a defender.”
Compact and Computationally Efficient Representation of Deep Neural Networks (2018, positive): “…Ω :[0 , 4 , 3 , 2] colI :[4 , 9 , 11 , 1 , 8 , 3 , 7 , 0 , 1 , 5 , 8 , 9 , 11 , 0 , 3 , 7 , 2 , 9 , 3 , 4 , 5 , 8 , 9 , 7 , 1 , 2 , 5 , 7] Ω Ptr :[0 , 3 , 5 , 7 , 13 , 16 , 17 , 18 , 23 , 24 , 28] rowPtr :[0 , 3 , 4 , 7 , 9 , 10] Notice that we can uniquely reconstruct M from this data structure.”
MUSIQ: Multi-scale Image Quality Transformer (2021, neutral): “…from [43] for the results of the reference methods. method SRCC PLCC BRISQUE [26] 0.665 0.681 ILNIQE [47] 0.507 0.523 HOSA [39] 0.671 0.694 BIECON [19] 0.618 0.651 WaDIQaM [3] 0.797 0.805 PQR [44] 0.880 0.884 SFA [21] 0.856 0.872 DBCNN [49] 0.875 0.884 MetaIQA [50] 0.850 0.887 BIQA [34] Table 2.”
RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution (2019, background): “[19, 4] integrate a general CNN framework which can predict image quality on local regions.”
Understanding Patch-Based Learning by Explaining Predictions (2018, background): “For applications where the input signal is very large in time or space, it has been a common practice to train the model on small patches or snippets of that signal [2,6,8].”
TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment (2023, background): “However, early end-to-end works [17], [18] suffer from over-fitting.”
Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, neutral): “Trustful segmentation results are hard to get for river ices.”
PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference (2018, methodology): “This network uses the feature difference from the last CONV layer of the FE subnet as input (denoted as ym R −ym A for A and R), since the weight for a patch is akin to the higher-level patch saliency [7, 56, 57] captured by deeper CONV layers.”
Image Quality Assessment: Unifying Structure and Texture Similarity (2020, methodology): “Recently, there has been a surge of interest in leveraging advances in large-scale optimization to develop data-driven IQA measures [7], [8], [13], [19].”
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective (2023, methodology): “, Gaussian noise and JPEG compression) are of primary concern, patchwise training [4], quality-aware pre-training [32, 37, 76], and learning from noisy pseudolabels [2, 38, 67] are practical training tricks with less (or no) reliance on MOSs.”
Domain Fingerprints for No-Reference Image Quality Assessment (2019, methodology): “4) Evaluation Metrics: Following most previous works [5], [6], [13], the performances on above datasets are evaluated by two common metrics for model evaluation: the Linear Correlation Coefﬁcient (LCC) and the Spearman’s Rank Order Correlation Coefﬁcient (SROCC).”
FusionDN: A Unified Densely Connected Network for Image Fusion (2020, methodology): “Complementarily, IQA can assess the noise and other problems of reducing image quality.”
MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment (2022, methodology): “[3,14,20,37,61] leveraged the extracted features to predict the subjective score with the help of pre-trained CNN-based models.”
MetaIQA: Deep Meta-Learning for No-Reference Image Quality Assessment (2020, methodology): “In recent years, the deep learning-based general-purpose NR-IQA methods have demonstrated superior prediction performance over traditional methods [1, 44, 55, 28, 2, 30, 52, 58, 57].”