No-Reference Image Quality Assessment via Transformers, Relative
Ranking, and Self-Consistency
- URL: http://arxiv.org/abs/2108.06858v1
- Date: Mon, 16 Aug 2021 02:07:08 GMT
- Title: No-Reference Image Quality Assessment via Transformers, Relative
Ranking, and Self-Consistency
- Authors: S. Alireza Golestaneh, Saba Dadsetan, Kris M. Kitani
- Abstract summary: The goal of No-Reference Image Quality Assessment (NR-IQA) is to estimate the perceptual image quality in accordance with subjective evaluations.
We propose a novel model to address the NR-IQA task by leveraging a hybrid approach that benefits from Convolutional Neural Networks (CNNs) and self-attention mechanism in Transformers.
- Score: 38.88541492121366
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The goal of No-Reference Image Quality Assessment (NR-IQA) is to estimate the
perceptual image quality in accordance with subjective evaluations, it is a
complex and unsolved problem due to the absence of the pristine reference
image. In this paper, we propose a novel model to address the NR-IQA task by
leveraging a hybrid approach that benefits from Convolutional Neural Networks
(CNNs) and self-attention mechanism in Transformers to extract both local and
non-local features from the input image. We capture local structure information
of the image via CNNs, then to circumvent the locality bias among the extracted
CNNs features and obtain a non-local representation of the image, we utilize
Transformers on the extracted features where we model them as a sequential
input to the Transformer model. Furthermore, to improve the monotonicity
correlation between the subjective and objective scores, we utilize the
relative distance information among the images within each batch and enforce
the relative ranking among them. Last but not least, we observe that the
performance of NR-IQA models degrades when we apply equivariant transformations
(e.g. horizontal flipping) to the inputs. Therefore, we propose a method that
leverages self-consistency as a source of self-supervision to improve the
robustness of NRIQA models. Specifically, we enforce self-consistency between
the outputs of our quality assessment model for each image and its
transformation (horizontally flipped) to utilize the rich self-supervisory
information and reduce the uncertainty of the model. To demonstrate the
effectiveness of our work, we evaluate it on seven standard IQA datasets (both
synthetic and authentic) and show that our model achieves state-of-the-art
results on various datasets.
Related papers
- Attention Down-Sampling Transformer, Relative Ranking and Self-Consistency for Blind Image Quality Assessment [17.04649536069553]
No-reference image quality assessment is a challenging domain that addresses estimating image quality without the original reference.
We introduce an improved mechanism to extract local and non-local information from images via different transformer encoders and CNNs.
A self-consistency approach to self-supervision is presented, explicitly addressing the degradation of no-reference image quality assessment (NR-IQA) models.
arXiv Detail & Related papers (2024-09-11T09:08:43Z) - Large Multi-modality Model Assisted AI-Generated Image Quality Assessment [53.182136445844904]
We introduce a large Multi-modality model Assisted AI-Generated Image Quality Assessment (MA-AGIQA) model.
It uses semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts.
It achieves state-of-the-art performance, and demonstrates its superior generalization capabilities on assessing the quality of AI-generated images.
arXiv Detail & Related papers (2024-04-27T02:40:36Z) - Transformer-based No-Reference Image Quality Assessment via Supervised
Contrastive Learning [36.695247860715874]
We propose a novel Contrastive Learning (SCL) and Transformer-based NR-IQA model SaTQA.
We first train a model on a large-scale synthetic dataset by SCL to extract degradation features of images with various distortion types and levels.
To further extract distortion information from images, we propose a backbone network incorporating the Multi-Stream Block (MSB) by combining the CNN inductive bias and Transformer long-term dependence modeling capability.
Experimental results on seven standard IQA datasets show that SaTQA outperforms the state-of-the-art methods for both synthetic and authentic datasets
arXiv Detail & Related papers (2023-12-12T06:01:41Z) - DeepDC: Deep Distance Correlation as a Perceptual Image Quality
Evaluator [53.57431705309919]
ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models.
We develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features.
We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets.
arXiv Detail & Related papers (2022-11-09T14:57:27Z) - CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms.
Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner.
Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
arXiv Detail & Related papers (2022-06-29T15:22:01Z) - Attentions Help CNNs See Better: Attention-based Hybrid Image Quality
Assessment Network [20.835800149919145]
Image quality assessment (IQA) algorithm aims to quantify the human perception of image quality.
There is a performance drop when assessing distortion images generated by generative adversarial network (GAN) with seemingly realistic texture.
We propose an Attention-based Hybrid Image Quality Assessment Network (AHIQ) to deal with the challenge and get better performance on the GAN-based IQA task.
arXiv Detail & Related papers (2022-04-22T03:59:18Z) - Learning Transformer Features for Image Quality Assessment [53.51379676690971]
We propose a unified IQA framework that utilizes CNN backbone and transformer encoder to extract features.
The proposed framework is compatible with both FR and NR modes and allows for a joint training scheme.
arXiv Detail & Related papers (2021-12-01T13:23:00Z) - Image Quality Assessment using Contrastive Learning [50.265638572116984]
We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem.
We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models.
Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
arXiv Detail & Related papers (2021-10-25T21:01:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.