Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective

By Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, Kede Ma. Computer Vision and Pattern Recognition, 2023

409 citations recorded by the source corpus.

We aim at advancing blind image quality assessment (BIQA), which predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks, in a way that the model parameter sharing and the loss weighting are determined automatically. Specifically, we first describe all candidate label combinations (from multiple tasks) using a textual template, and compute the joint probability from the cosine similarities of the visual-textual embeddings. Predictions of each task can be inferred from the joint distribution, and optimized by carefully designed loss functions. Through comprehensive experiments on learning three tasks - BIQA, scene classification, and distortion type identification, we verify that the proposed BIQA method 1) benefits from the scene classification and distortion type identification tasks and outperforms the state-of-the-art on multiple IQA datasets, 2) is more robust in the group maximum differentiation competition, and 3) realigns the quality annotations from different IQA datasets more effectively. The source code is available at https://github.com/zwx8981/LIQE.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/230cf347b3d12c7506bbf9a34fa0588aac66c353

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Haoning Wu
2. Qiong Yan
3. Weisi Lin
4. Jingwen Hou
5. Liang Liao
6. Jiadi Mo
7. Chaofeng Chen
8. Wenxiu Sun

In Media Technology

1. Haoning Wu
2. Qiong Yan
3. Weisi Lin
4. Jingwen Hou
5. Liang Liao
6. Jiadi Mo
7. Chaofeng Chen
8. Wenxiu Sun

Citations in the corpus

• TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment (2023, background): “Latest works explore the utilization of vision-language models in IQA [68], [69], as well as the quality assessment of text-to-image models [70].”