Blind Image Quality Assessment Based on High Order Statistics Aggregation

By Jingtao Xu, Peng Ye, Qiaohong Li, Haiqing Du, Yong Liu, David S. Doermann. IEEE Transactions on Image Processing, 2016

473 citations recorded by the source corpus.

Blind image quality assessment (BIQA) research aims to develop a perceptual model to evaluate the quality of distorted images automatically and accurately without access to the non-distorted reference images. The state-of-the-art general purpose BIQA methods can be classified into two categories according to the types of features used. The first includes handcrafted features which rely on the statistical regularities of natural images. These, however, are not suitable for images containing text and artificial graphics. The second includes learning-based features which invariably require large codebook or supervised codebook updating procedures to obtain satisfactory performance. These are time-consuming and not applicable in practice. In this paper, we propose a novel general purpose BIQA method based on high order statistics aggregation (HOSA), requiring only a small codebook. HOSA consists of three steps. First, local normalized image patches are extracted as local features through a regular grid, and a codebook containing 100 codewords is constructed by K-means clustering. In addition to the mean of each cluster, the diagonal covariance and coskewness (i.e., dimension-wise variance and skewness) of clusters are also calculated. Second, each local feature is softly assigned to several nearest clusters and the differences of high order statistics (mean, variance and skewness) between local features and corresponding clusters are softly aggregated to build the global quality aware image representation. Finally, support vector regression is adopted to learn the mapping between perceptual features and subjective opinion scores. The proposed method has been extensively evaluated on ten image databases with both simulated and realistic image distortions, and shows highly competitive performance to the state-of-the-art BIQA methods.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/612fe32a9357186f0769e7c849684fd87ee908fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

KADID-10k: A Large-scale Artificially Distorted IQA Database (2019, unknown): “…0.431 0.299 BLIINDS-II [25] 0.559 0.527 0.375 BRISQUE [26] 0.554 0.519 0.368 CORNIA * [27] 0.580 0.541 0.384 DIIVINE [28] 0.532 0.489 0.341 HOSA * [29] 0.653 0.609 0.438 SSEQ [30] 0.463 0.424 0.295 InceptionResNetV2 (ﬁne-tune) 0.734 0.731 0.546 * To reduce time complexity, CORNIA and HOSA…”
MetaIQA: Deep Meta-Learning for No-Reference Image Quality Assessment (2020, methodology): “Since the distortion types are usually not known in advance in real-world applications, more and more attention has been drawn in general-purpose NR-IQA methods over the past few years [39, 32, 56, 51, 50, 12, 54, 49, 10].”
Deep CNN-Based Blind Image Quality Predictor (2019, methodology): “Meanwhile, in [15] and [16], dictionary learning was adapted to capture effective features from the raw patches.”
From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality (2019, background): “No-reference (NR) or blind picture quality prediction is largely unsolved, though popular models exist [14, 15, 16, 17, 18, 19, 20].”
Blind Quality Assessment Based on Pseudo-Reference Image (2018, background): “CORNIA [36] and HOSA [37] are both based on unsupervised feature learning.”
NIMA: Neural Image Assessment (2017, background): “Correlation between ground truth and results of NIMA(VGG16) are close to the state-of-the-art results in [35] and [7].”
On the use of deep learning for blind image quality assessment (2016, methodology): “and median SROCC across 100 trainval-test random splits of the TID2013. Method LCC SROCC DIIVINE [34] 0.89 0.88 BRISQUE [31] 0.92 0.89 BLIINDS-II [39] 0.91 0.88 Low Level Features [21] 0.89 0.88 HOSA [49] 0.96 0.95 DeepBIQ 0.96 0.96 extracted from pre-trained Convolutional Neural Networks (CNNs) as ”
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective (2023, methodology): “Conventional BIQA either relied on hand-engineered features in the form of natural scene statistics (NSS) [12, 44–46] or shallow feature learning [18, 69, 71] in the form of codebooks.”
Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network (2019, methodology): “In this subsection, we conduct cross database validations to compare the generalizability of DB-CNN against BRISQUE [7], M3 [48], FRIQUEE [22], CORNIA [8], and HOSA [49].”
HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation (2019, background): “However, for the real world applications, the deep features are always a complex, non-Gaussian distribution, which can not be completely characterized by its first-order or second-order statistics (Jia and Darrell 2011; Xu et al. 2016).”
RankIQA: Learning from Rankings for No-Reference Image Quality Assessment (2017, methodology): “For all distortions, our method RankIQA+FT achieves about 5% higher than HOSA.”
No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency (2021, unknown): “Before the raise of deep learning, early version of general-purpose NR-IQA methods mainly divide into natu-ral scene statistics (NSS) based metrics [16, 17, 6, 5, 18, 19, 8, 20, 21] and learning-based metrics [22, 23, 24, 25, 26, 27].”
Blind Image Quality Estimation via Distortion Aggravation (2018, background): “Ten state-of-the-art blind IQA measures act as competitors in this paper, including: (1) NSS based measures, i.e., DIIVINE [15], BLIINDS2 [16], BRISQUE [17], NIQE [28], and ILNIQE [19]; (2) learning based measures, i.e., CORNIA [29] and HOSA [30]; (3) human vision based and some other measures, i.e.”
UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2020, methodology): “baseline models include NIQE [53], ILNIQE [78], VIIDEO [55], BRISQUE [38], GM-LOG [57], HIGRADE [40], FRIQUEE [41], CORNIA [43], HOSA [61], KonCept512 [70], PaQ-2-PiQ [64], V-BLIINDS [39], and TLVQM [42].”
KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment (2019, methodology): “According to the availability of pristine reference images, objective IQA methods are categorized as Full-Reference (FR), e.g., SSIM [5], Reduced-Reference (RR), e.g., RRED [6], and No-Reference (NR), e.g., CORNIA [7], HOSA [8].”