Quality Assessment of In-the-Wild Videos

By Dingquan Li, Tingting Jiang, Ming Jiang. ACM Multimedia, 2019

420 citations recorded by the source corpus.

Quality assessment of in-the-wild videos is a challenging problem because of the absence of reference videos and shooting distortions. Knowledge of the human visual system can help establish methods for objective quality assessment of in-the-wild videos. In this work, we show two eminent effects of the human visual system, namely, content-dependency and temporal-memory effects, could be used for this purpose. We propose an objective no-reference video quality assessment method by integrating both effects into a deep neural network. For content-dependency, we extract features from a pre-trained image classification neural network for its inherent content-aware property. For temporal-memory effects, long-term dependencies, especially the temporal hysteresis, are integrated into the network with a gated recurrent unit and a subjectively-inspired temporal pooling layer. To validate the performance of our method, experiments are conducted on three publicly available in-the-wild video quality assessment databases: KoNViD-1k, CVD2014, and LIVE-Qualcomm, respectively. Experimental results demonstrate that our proposed method outperforms five state-of-the-art methods by a large margin, specifically, 12.39%, 15.71%, 15.45%, and 18.09% overall performance improvements over the second-best method VBLIINDS, in terms of SROCC, KROCC, PLCC and RMSE, respectively. Moreover, the ablation study verifies the crucial role of both the content-aware features and the modeling of temporal-memory effects. The PyTorch implementation of our method is released at https://github.com/lidq92/VSFA.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2eebdd917d167aefcb7384ab8cd62599588662ed

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives (2022, background): “On the other hand, several recent studies [2,6,7,20,21] notice that preferences on non-technical semantic factors ( e.g. , contents, composition ) also affect human quality assessment on UGC videos.”
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels (2023, methodology): “Similar as IQA, while traditional approaches on VQA are typically based on handcraft features, e.g. TLVQM (Korhonen, 2019), VIDEVAL (Tu et al., 2021a), and RAPIQUE (Tu et al., 2021b), recent deep-learning-based methods, such as VSFA (Li et al., 2019), BVQA (Li et al., 2022), Dis-CoVQA (Wu et al.) an”
FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling (2022, methodology): “In recent years, deep-learning-based VQA methods [22,23,40,8,42,21] have shown better performance on in-the-wild VQA benchmarks [32,12,38,40].”
VBench: Comprehensive Benchmark Suite for Video Generative Models (2023, methodology): “Meanwhile, the Video Quality Assessment (VQA) methods [63, 94, 108–114] are primarily designed for real videos, thereby neglecting the unique challenges posed by generative models, such as artifacts in synthesized videos.”
UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2020, methodology): “nnected layers to predict video quality. Zhang [69] leveraged transfer learning to develope a general-purpose BVQA framework based on weakly supervised learning and a resampling strategy. In the VSFA [70] model, the authors applied a pre-trained image classiﬁcation CNN as a deep feature extractor an”