Learning a No-Reference Quality Metric for Single-Image Super-Resolution

By Chao Ma, Chih-Yuan Yang, Xiaokang Yang, Ming-Hsuan Yang. 2016

607 citations recorded by the source corpus.

Abstract Numerous single-image super-resolution algorithms have been proposed in the literature, but few studies address the problem of performance evaluation based on visual perception. While most super-resolution images are evaluated by full-reference metrics, the effectiveness is not clear and the required ground-truth images are not always available in practice. To address these problems, we conduct human subject studies using a large set of super-resolution images and propose a no-reference metric learned from visual perceptual scores. Specifically, we design three types of low-level statistical features in both spatial and frequency domains to quantify super-resolved artifacts, and learn a two-stage regression model to predict the quality scores of super-resolution images without referring to ground-truth images. Extensive experimental results show that the proposed metric is effective and efficient to assess the quality of super-resolution images based on human perception.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Radu Timofte
3. Chen Change Loy
4. Ming-Hsuan Yang
5. Lei Zhang
6. Dacheng Tao
7. Yu Qiao
8. Chao Dong

In Media Technology

1. Radu Timofte
2. Luc Van Gool
3. Chao Dong
4. Kai Zhang
5. Chen Change Loy
6. Saeed Anwar
7. Nick Barnes
8. Jinjin Gu

Citations in the corpus

• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, methodology): “Since the ground-truth for the RealSRSet dataset is not available, we adopt the non-reference image quality assessment (IQA) metrics including NIQE [38], NRQM [32] and PI [4] for quantitative evaluation.”
• The Perception-Distortion Tradeoff (2017, methodology): “[25, 26, 33, 27, 45, 16, 23], which quantify the perceptual quality of an image without a corresponding original image.”
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks (2018, methodology): “The perceptual quality is judged by the non-reference measures of Ma’s score [23] and NIQE [24], i.e., perceptual index = ((10 − Ma)+NIQE).”
• TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment (2023, methodology): “Traditional approaches for NR-IQA rely on natural scene statistics (NSS) [6], [29], [30], [31], [32], [33].”
• Analyzing Perception-Distortion Tradeoff using Enhanced Perceptual Super-resolution Network (2018, methodology): “PI is computed by combining the quality measures of Ma-score [32] and NIQE [36] as follows Note that, a lower PI indicates better perceptual quality.”
• Learning to Maintain Natural Image Statistics (2018, methodology): “SSIM [37] measures structural similarity to the ground-truth, while NRQM [45] measures no-reference perceptual quality.”
• Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement (2020, methodology): “In addition to the US score, we also employ a non-reference perceptual index (PI) [1, 17, 21] to evaluate the perceptual quality of our results.”
• RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution (2019, methodology): “To better understand the effects, we provide the histograms of NIQE [6]/Ma [4] label value in the validation dataset of the rank dataset.”
• Enhanced Pix2pix Dehazing Network (2019, background): “2 ((10−Ma) +NIQE), where Ma and NIQE are two image qualification indexes which are detailed in [4] and [13].”
• 2018 PIRM Challenge on Perceptual Image Super-resolution (2018, methodology): “[22], NIQE [27], BRISQUE [26] and the PI defined by (1).”
• Creating High Resolution Images with a Latent Adversarial Generator (2020, background): “Other metrics [16, 12] have been proposed to measure the perceptual quality of a single image.”
• Exploring CLIP for Assessing the Look and Feel of Images (2022, methodology): “Conventional NR-IQA methods [29, 31, 32, 45, 66] mainly rely on natural scene statistics (NSS) [44], a model revealing statistic property of natural images.”
• A Deep Journey into Super-resolution (2019, methodology): “The challenge evaluation is based on traditional full-reference metrics such as PSNR, SSIM, RMSE, FC [95], LPIPS [96], as well as the no-reference methods by Ma et al. [97], NIQE [98], BRISQUE [99].”
• Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation (2021, methodology): “perceptual index (PI) [44], [45], [46] to evaluate the TABLE 3 The Statistic Relations Between Enhancement Performance and Input Sizes Measured in PSNR and FLOPs”
• The 2018 PIRM Challenge on Perceptual Image Super-Resolution (2018, methodology): “Super-resolution algorithms, plotted according to the mean reconstruction accuracy (measured by RMSE values) and mean perceptual quality (measured by the recent metric [22]).”
• Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, neutral): “NRQM [27] is a learned IQA score.”