NIMA: Neural Image Assessment

By Hossein Talebi, Peyman Milanfar. 2017

1,211 citations recorded by the source corpus.

Automatically learned quality assessment for images has recently become a hot topic due to its usefulness in a wide variety of applications, such as evaluating image capture pipelines, storage techniques, and sharing media. Despite the subjective nature of this problem, most existing methods only predict the mean opinion score provided by data sets, such as AVA and TID2013. Our approach differs from others in that we predict the distribution of human opinion scores using a convolutional neural network. Our architecture also has the advantage of being significantly simpler than other methods with comparable performance. Our proposed approach relies on the success (and retraining) of proven, state-of-the-art deep object recognition networks. Our resulting network can be used to not only score images reliably and with high correlation to human perception, but also to assist with adaptation and optimization of photo editing/enhancement algorithms in a photographic pipeline. All this is done without need for a “golden” reference image, consequently allowing for single-image, semantic- and perceptually-aware, no-reference quality assessment.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (2018, background): “Concurrent work by Talebi and Milanfar [53, 54] train a deep network in the context of NR-IQA for image aesthetics.”
Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives (2022, background): “The downsampling strategies are widely applied in many existing state-of-the-art aesthetic assessment methods [24, 25, 27, 65, 66], further proving that they are able to preserve aesthetic information in visual contents.”
MUSIQ: Multi-scale Image Quality Transformer (2021, methodology): “In the deep learning era, many IQA approaches [12, 34, 36, 43, 49] have achieved signiﬁ-cant success by leveraging the power of convolutional neural networks (CNNs).”
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels (2023, methodology): “To address this need, from handcraft approaches (Mit-tal et al., 2013; 2012) to deep-neural-network-based meth-ods (Talebi & Milanfar, 2018; Zhang et al., 2020; Ke et al., 2021), the endeavor to improve the accuracies of visual assessors never stops.”
MaxViT: Multi-Axis Vision Transformer (2022, methodology): “We followed [67] and used the normalized Earth Mover’s Distance as our training loss.”
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections (2020, methodology): “As a coarse pre-ﬁltering step, we remove low quality images largely consisting of transient objects by dropping the lowest portion of images as ranked automatically by NIMA [32], using a minimum score threshold of 3 .”
MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment (2022, methodology): “With the success of deep learning and Convolution Neural Network (CNN) in many computer vision tasks, CNN-based methods have significantly outperformed previous approaches in terms of real-world distortions by directly extracting discriminative features from LQ images [1,3,14,20,26,37,39,45,59,61].”
A Survey of Human-in-the-loop for Machine Learning (2021, background): “The research field of image enhancement has attracted ample attention from researchers in recent years, especially after the emergence of deep neural network algorithms [146, 147, 148].”
Deep Learning for Image Super-Resolution: A Survey (2019, background): “[67] propose no-reference Ma and NIMA, respectively, which are learned from visual perceptual scores and directly predict the quality scores without ground-truth images.”
PSD: Principled Synthetic-to-Real Dehazing Guided by Physical Priors (2021, methodology): “Also, we employ three well-known no-reference image quality assessment indicators: BRISQUE [25], NIQE [26], and NIMA [36].”
From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality (2019, methodology): “The Baseline, RoIPool, and Feedback Models all outperformed the shallow and other deep models [46, 65] on both datasets.”
No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency (2021, background): “Talebi et al. [37] proposed a CNN-based model to predict the perceptual distribution of subjective quality scores (instead of the mean value).”
TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment (2023, methodology): “2) MOS distribution labels: For datasets that are labeled with score distributions, such as the AVA dataset [25], we predict the distribution and use the Earth Mover’s Distance (EMD) loss proposed by [43].”
KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment (2019, methodology): “Talebi et al. [43] introduced the Earth Mover’s Distance (EMD) loss in their work on BIQA, suggesting an improved performance over cross-entropy.”
MetaIQA: Deep Meta-Learning for No-Reference Image Quality Assessment (2020, methodology): “In recent years, the deep learning-based general-purpose NR-IQA methods have demonstrated superior prediction performance over traditional methods [1, 44, 55, 28, 2, 30, 52, 58, 57].”
Learned perceptual image enhancement (2017, methodology): “Our image quality predictor is built on image classifier architectures [34].”
Creating High Resolution Images with a Latent Adversarial Generator (2020, background): “Other metrics [16, 12] have been proposed to measure the perceptual quality of a single image.”