MUSIQ: Multi-scale Image Quality Transformer

By Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, Feng Yang. IEEE International Conference on Computer Vision, 2021

1,517 citations recorded by the source corpus.

Image quality assessment (IQA) is an important research topic for understanding and improving visual experience. The current state-of-the-art IQA methods are based on convolutional neural networks (CNNs). The performance of CNN-based models is often compromised by the fixed shape constraint in batch training. To accommodate this, the input images are usually resized and cropped to a fixed shape, causing image quality degradation. To address this, we design a multi-scale image quality Transformer (MUSIQ) to process native resolution images with varying sizes and aspect ratios. With a multi-scale image representation, our proposed method can capture image quality at different granularities. Furthermore, a novel hash-based 2D spatial embedding and a scale embedding is proposed to support the positional embedding in the multi-scale representation. Experimental results verify that our method can achieve state-of-the-art performance on multiple large scale IQA datasets such as PaQ-2-PiQ [41], SPAQ [11], and KonIQ-10k [16]. 1

View Paper: https://www.semanticscholar.org/paper/e3d06054af531ee2f42270d43100b309c28546ef

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “4 Evaluation on Real-World Data Table 4 lists the comparative evaluation using CLIPIQA [66] and MUSIQ [65] of various methods on two real-world datasets.”
Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels (2023, methodology): “To address this need, from handcraft approaches (Mit-tal et al., 2013; 2012) to deep-neural-network-based meth-ods (Talebi & Milanfar, 2018; Zhang et al., 2020; Ke et al., 2021), the endeavor to improve the accuracies of visual assessors never stops.”
VBench: Comprehensive Benchmark Suite for Video Generative Models (2023, methodology): “Imaging quality refers to the distortion (e.g., over-exposure, noise, blur) presented in the generated frames, and we evaluate it us-ing the MUSIQ [57] image quality predictor trained on the SPAQ [24] dataset.”
MaxViT: Multi-Axis Vision Transformer (2022, methodology): “Similar to [41,75], we split the dataset into train and test sets, such that 20% of the data is used for testing.”
Towards Robust Blind Face Restoration with Codebook Lookup Transformer (2022, methodology): “For the evaluation on real-world datasets without ground truth, we employ the widely-used non-reference perceptual metrics: FID [15] and MUSIQ (KonIQ) [22].”
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, methodology): “To better evaluate the quality , we include several no-reference image quality assessment (IQA) metrics: MANIQA[64], MUSIQ [24] and CLIP-IQA [51].”
FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling (2022, background): “Naive approaches such as resizing [17,43] and cropping [14,15] cannot preserve video quality well.”
Exploring CLIP for Assessing the Look and Feel of Images (2022, methodology): “…performance of our CLIP-IQA, we adopt Spearman’s Rank-order Correlation Coefﬁcient (SROCC) and Pearson’s Linear Correlation Coefﬁcient (PLCC), which are widely used for evaluating the performance of IQA methods (Bosse et al. 2017; Su et al. 2020; Zhu et al. 2020; Ke et al. 2021; Pan et al. 2022).”
MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment (2022, methodology): “MUSIQ [15] used Transformer to encode 3 scales of distortion image features and settled the incompatibility of different input image sizes during training and testing.”
OminiControl: Minimal and Universal Control for Diffusion Transformer (2024, unknown): “Generation quality is measured using FID [16], SSIM, CLIP-IQA[53], MAN-IQA [59], MUSIQ [21], and PSNR[7] for visual fidelity, along with CLIP Text and CLIP Image scores [43] for consistency.”
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective (2023, methodology): “For competing models, we either directly adopt the publicly available implementations [4, 17, 19, 72], or re-train them on our datasets with the training codes provided by the respective authors [13,55,76,78].”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “We selected the following metrics for quantitative comparison: full-reference metrics PSNR, SSIM, LPIPS [93], and the non-reference metrics ManIQA [83], ClipIQA [74], MUSIQ [43].”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, unknown): “To verify the effectiveness of our approach, we compare our StableSR with several state-of-the-art methods 3 , i.e ., RealSR 4 [29], BSRGAN [76], Real-ESRGAN+ [63], DASR [37], FeMaSR [8], and latent diffusion model (LDM) [50].”
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, background): “2022) and MUSIQ (Ke et al. 2021), to compare the competing Real-ISR models.”
SinSR: Diffusion-Based Image Super-Resolution in a Single Step (2023, methodology): “Besides, two recent SOTA non-reference metrics are used to justify the realism of all the images, i.e. , CLIP-IQA [39] which leverages a CLIP model [31] pre-trained on a large-scale dataset (Laion400M [36]) and MUSIQ [15].”
TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment (2023, methodology): “2) More results on KonIQ-10k: Following previous works [37], [45], we report the results of 10 random splits on KonIQ10k in Tab.”
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, unknown): “NIQE [71], MANIQA [64], MUSIQ [64], and CLIPIQA [64] are no-reference image quality measures.”