Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images

• URL: http://arxiv.org/abs/2303.10369v1
• Date: Sat, 18 Mar 2023 09:04:55 GMT
• Title: Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images
• Authors: Miaohui Wang, Zhuowei Xu, Mai Xu, and Weisi Lin
• Abstract summary: Blind image quality assessment (BIQA) aims at automatically and accurately forecasting objective scores for visual signals. Recent developments in this field are dominated by unimodal solutions inconsistent with human subjective rating patterns. We present a unique blind multimodal quality assessment (BMQA) of low-light images from subjective evaluation to objective score.
• Score: 73.27643795557778
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Blind image quality assessment (BIQA) aims at automatically and accurately forecasting objective scores for visual signals, which has been widely used to monitor product and service quality in low-light applications, covering smartphone photography, video surveillance, autonomous driving, etc. Recent developments in this field are dominated by unimodal solutions inconsistent with human subjective rating patterns, where human visual perception is simultaneously reflected by multiple sensory information (e.g., sight and hearing). In this article, we present a unique blind multimodal quality assessment (BMQA) of low-light images from subjective evaluation to objective score. To investigate the multimodal mechanism, we first establish a multimodal low-light image quality (MLIQ) database with authentic low-light distortions, containing image and audio modality pairs. Further, we specially design the key modules of BMQA, considering multimodal quality representation, latent feature alignment and fusion, and hybrid self-supervised and supervised learning. Extensive experiments show that our BMQA yields state-of-the-art accuracy on the proposed MLIQ benchmark database. In particular, we also build an independent single-image modality Dark-4K database, which is used to verify its applicability and generalization performance in mainstream unimodal applications. Qualitative and quantitative results on Dark-4K show that BMQA achieves superior performance to existing BIQA approaches as long as a pre-trained quality semantic description model is provided. The proposed framework and two databases as well as the collected BIQA methods and evaluation metrics are made publicly available.

Related papers

• Q-Ground: Image Quality Grounding with Large Multi-modality Models [61.72022069880346]
  We introduce Q-Ground, the first framework aimed at tackling fine-scale visual quality grounding. Q-Ground combines large multi-modality models with detailed visual quality analysis. Central to our contribution is the introduction of the QGround-100K dataset.
  arXiv  Detail & Related papers  (2024-07-24T06:42:46Z)
• Descriptive Image Quality Assessment in the Wild [25.503311093471076]
  VLM-based Image Quality Assessment (IQA) seeks to describe image quality linguistically to align with human expression. We introduce Depicted image Quality Assessment in the Wild (DepictQA-Wild) Our method includes a multi-functional IQA task paradigm that encompasses both assessment and comparison tasks, brief and detailed responses, full-reference and non-reference scenarios.
  arXiv  Detail & Related papers  (2024-05-29T07:49:15Z)
• Opinion-Unaware Blind Image Quality Assessment using Multi-Scale Deep Feature Statistics [54.08757792080732]
  We propose integrating deep features from pre-trained visual models with a statistical analysis model to achieve opinion-unaware BIQA (OU-BIQA) Our proposed model exhibits superior consistency with human visual perception compared to state-of-the-art BIQA models.
  arXiv  Detail & Related papers  (2024-05-29T06:09:34Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models [28.194638379354252]
  We introduce a Depicted image Quality Assessment method (DepictQA), overcoming the constraints of traditional score-based methods. DepictQA allows for detailed, language-based, human-like evaluation of image quality by leveraging Multi-modal Large Language Models. These results showcase the research potential of multi-modal IQA methods.
  arXiv  Detail & Related papers  (2023-12-14T14:10:02Z)
• Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment [50.82681686110528]
  Blind Omnidirectional Image Quality Assessment (BOIQA) aims to objectively assess the human perceptual quality of omnidirectional images (ODIs) The quality assessment of ODIs is severely hampered by the fact that the existing BOIQA pipeline lacks the modeling of the observer's browsing process. We propose a novel multi-sequence network for BOIQA called Assessor360, which is derived from the realistic multi-assessor ODI quality assessment procedure.
  arXiv  Detail & Related papers  (2023-05-18T13:55:28Z)
• UNO-QA: An Unsupervised Anomaly-Aware Framework with Test-Time Clustering for OCTA Image Quality Assessment [4.901218498977952]
  We propose an unsupervised anomaly-aware framework with test-time clustering for optical coherence tomography angiography ( OCTA) image quality assessment. A feature-embedding-based low-quality representation module is proposed to quantify the quality of OCTA images. We perform dimension reduction and clustering of multi-scale image features extracted by the trained OCTA quality representation network.
  arXiv  Detail & Related papers  (2022-12-20T18:48:04Z)
• No-Reference Image Quality Assessment via Feature Fusion and Multi-Task Learning [29.19484863898778]
  Blind or no-reference image quality assessment (NR-IQA) is a fundamental, unsolved, and yet challenging problem. We propose a simple and yet effective general-purpose no-reference (NR) image quality assessment framework based on multi-task learning. Our model employs distortion types as well as subjective human scores to predict image quality.
  arXiv  Detail & Related papers  (2020-06-06T05:04:10Z)
• Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild [98.48284827503409]
  We develop a textitunified BIQA model and an approach of training it for both synthetic and realistic distortions. We employ the fidelity loss to optimize a deep neural network for BIQA over a large number of such image pairs. Experiments on six IQA databases show the promise of the learned method in blindly assessing image quality in the laboratory and wild.
  arXiv  Detail & Related papers  (2020-05-28T13:35:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.