PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference

By Ekta Prashnani, Hong Cai, Yasamin Mostofi, Pradeep Sen. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

376 citations recorded by the source corpus.

The ability to estimate the perceptual error between images is an important problem in computer vision with many applications. Although it has been studied extensively, however, no method currently exists that can robustly predict visual differences like humans. Some previous approaches used hand-coded models, but they fail to model the complexity of the human visual system. Others used machine learning to train models on human-labeled datasets, but creating large, high-quality datasets is difficult because people are unable to assign consistent error labels to distorted images. In this paper, we present a new learning-based method that is the first to predict perceptual image error like human observers. Since it is much easier for people to compare two given images and identify the one more similar to a reference than to assign quality scores to each, we propose a new, large-scale dataset labeled with the probability that humans will prefer one image over another. We then train a deep-learning model using a novel, pairwise-learning framework to predict the preference of one distorted image over the other. Our key observation is that our trained network can then be used separately with only one distorted image and a reference to predict its perceptual error, without ever being trained on explicit human perceptual-error labels. The perceptual error estimated by our new metric, PieAPP, is well-correlated with human opinion. Furthermore, it significantly outperforms existing algorithms, beating the state-of-the-art by almost 3Ã— on our test set in terms of binary error rate, while also generalizing to new kinds of distortions, unlike previous learning-based methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a41175002926ccc7754c0fb3c989ff38846434f1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment (2023, methodology): “For FR datasets, we have LIVE [75], CSIQ [15], TID2013 [1], KADID-10k [76], PieAPP [22], BAPPS [8] and PIPAL [23].”
PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration (2020, methodology): “ge the gap between the results of IQA methods and human judgments. Similar to other computer vision problems, advanced data-driven methods have also motivated the investigation of applications of IQA [60,37]. In addition to the above FR-IQA methods, NR-IQA methods are proposed to assess image qualit”
Image Quality Assessment: Unifying Structure and Texture Similarity (2020, methodology): “Results, reported in Table 1, demonstrate that DISTS performs favorably in comparison to both classic methods ( e . g ., PSNR and SSIM [2]) and CNN-based models (Deep-IQA, PieAPP, and LPIPS).”
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data (2023, methodology): “Further tuning on the perceptual data yields improvements, such as in LPIPS [94], PIE-APP [65], or DPAM in the audio domain [53, 54].”
Exploring CLIP for Assessing the Look and Feel of Images (2022, methodology): “The main limitation of FR-IQA is that it is conﬁned to scenarios where references exist.”
A Deep Journey into Super-resolution (2019, neutral): “, LPIPS [105] and PieAPP [106].”