Image database TID 2013 : Peculiarities , results and perspectives

By Nikolay N. Ponomarenko, Lina Jin, Oleg Ieremeiev, Vladimir Vasilyevich Lukin, Karen O. Egiazarian, Karen, Egiazarian, Jaakko T. Astola, Benoit Vozel, Kacem Chehdi and 3 more. 2016

1,275 citations recorded by the source corpus.

This paper describes a recently created image database, TID2013, intended for evaluation of full-reference visual quality assessment metrics. With respect to TID2008, the new database contains a larger number (3000) of test images obtained from 25 reference images, 24 types of distortions for each reference image, and 5 levels for each type of distortion. Motivations for introducing 7 new types of distortions and one additional level of distortions are given; examples of distorted images are presented. Mean opinion scores (MOS) for the new database have been collected by performing 985 subjective experiments with volunteers (observers) from five countries (Finland, France, Italy, Ukraine, and USA). The availability of MOS allows the use of the designed database as a fundamental tool for assessing the effectiveness of visual quality. Furthermore, existing visual quality metrics have been tested with the proposed database and the collected results have been analyzed using rank order correlation coefficients between MOS and considered metrics. These correlation indices have been obtained both considering the full set of distorted images and specific image subsets, for highlighting advantages and drawbacks of existing, state of the art, quality metrics. Approaches to thorough performance analysis for a given metric are presented to detect practical situations or distortion types for which this metric is not adequate enough to human perception. The created image database and the collected MOS values are freely available for downloading and utilization for scientific purposes. We have created a new large database.This database contains larger number of distorted images and distortion types.MOS values for all images are obtained and provided.Analysis of correlation between MOS and a wide set of existing metrics is carried out.Methodology for determining drawbacks of existing visual quality metrics is described.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Evaluation & Benchmarks

Citations in the corpus

Blind Quality Assessment Based on Pseudo-Reference Image (2018, methodology): “1) Databases: Five large IQA databases are used as testbeds, including LIVE [9], TID2013 [10], CSIQ [11], SIQAD [12] and CCT [13].”
Group MAD Competition? A New Methodology to Compare Objective Image Quality Models (2016, methodology): “By contrast, the current database we created in this work contains 190 times more source images and 30 times more distorted images, respectively.”
A Haar wavelet-based perceptual similarity index for image quality assessment (2016, methodology): “The parameters C and α were selected to maximize the mean of the four Spearman rank order correlation coefficients (SROCC) obtained from comparing HaarPSIC and MOS values from subsets of the TID 2008 [26], TID 2013 [27], LIVE [28] and CSIQ [8] image databases.”
Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network (2019, methodology): “During fine-tuning of DB-CNN, we again adopt Adam [45] with a learning rate of 10−6 for LIVE [15] and CSIQ [42], 10−5 for TID2013 [16], LIVE MD [43] and LIVE Challenge [14], respectively.”
PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference (2018, methodology): “However, unlike the TID datasets [42, 43], we do not explicitly convert this preference into a quality score, since approaches such as Swiss tournaments introduce errors and do not scale.”
Visual Reinforcement Learning with Imagined Goals (2018, background): “In particular, pixel-wise Euclidean distance is not an effective reward function for visual tasks since distances between images do not correspond to meaningful distances between states [36, 49].”
Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data (2018, background): “We just choose CID2013, CCID2014, TID2008, CSIQ, TID2013 and SIQAD datasets that meet the requirement of conducting the leave-one-out cross-validation.”
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection (2020, neutral): “Specifically, as shown in Table 2, seven types of distortions defined in Image Quality Assessment (IQA) [27, 31] are included.”
Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild (2020, methodology): “Being exposed to more synthetic distortion types in TID2013 [43], deepIQA and RankIQA achieve better performance on KADID-10K [6] than MEON.”
Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, neutral): “Since each module is only responsible for obtaining relatively few new feature maps, the calculation cost can be greatly reduced and the prediction speed is improved.”
Exploring CLIP for Assessing the Look and Feel of Images (2022, methodology): “Table 5: Comparison to NR-IQA methods on synthetic dataset TID2013 (Ponomarenko et al. 2015).”
Domain Fingerprints for No-Reference Image Quality Assessment (2019, methodology): “We pre-train DA-IQA on Waterloo Exploration [39] and perform cross validation on TID2013 [40] and LIVE [41].”