Mask Reference Image Quality Assessment

• URL: http://arxiv.org/abs/2302.13770v2
• Date: Sun, 19 Mar 2023 14:14:29 GMT
• Title: Mask Reference Image Quality Assessment
• Authors: Pengxiang Xiao, Shuai He, Limin Liu, Anlong Ming
• Abstract summary: Mask Reference IQA (MR-IQA) is a method that masks specific patches of a distorted image and supplements missing patches with the reference image patches. Our method achieves state-of-the-art performances on the benchmark KADID-10k, LIVE and CSIQ datasets.
• Score: 8.087355843192109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding semantic information is an essential step in knowing what is being learned in both full-reference (FR) and no-reference (NR) image quality assessment (IQA) methods. However, especially for many severely distorted images, even if there is an undistorted image as a reference (FR-IQA), it is difficult to perceive the lost semantic and texture information of distorted images directly. In this paper, we propose a Mask Reference IQA (MR-IQA) method that masks specific patches of a distorted image and supplements missing patches with the reference image patches. In this way, our model only needs to input the reconstructed image for quality assessment. First, we design a mask generator to select the best candidate patches from reference images and supplement the lost semantic information in distorted images, thus providing more reference for quality assessment; in addition, the different masked patches imply different data augmentations, which favors model training and reduces overfitting. Second, we provide a Mask Reference Network (MRNet): the dedicated modules can prevent disturbances due to masked patches and help eliminate the patch discontinuity in the reconstructed image. Our method achieves state-of-the-art performances on the benchmark KADID-10k, LIVE and CSIQ datasets and has better generalization performance across datasets. The code and results are available in the supplementary material.

Related papers

• Reference-Free Image Quality Metric for Degradation and Reconstruction Artifacts [2.5282283486446753]
  We develop a reference-free quality evaluation network, dubbed "Quality Factor (QF) Predictor" Our QF Predictor is a lightweight, fully convolutional network comprising seven layers. It receives JPEG compressed image patch with a random QF as input, is trained to accurately predict the corresponding QF.
  arXiv  Detail & Related papers  (2024-05-01T22:28:18Z)
• PAME: Self-Supervised Masked Autoencoder for No-Reference Point Cloud Quality Assessment [34.256276774430575]
  No-reference point cloud quality assessment (NR-PCQA) aims to automatically predict the perceptual quality of point clouds without reference. We propose a self-supervised pre-training framework using masked autoencoders (PAME) to help the model learn useful representations without labels. Our method outperforms the state-of-the-art NR-PCQA methods on popular benchmarks in terms of prediction accuracy and generalizability.
  arXiv  Detail & Related papers  (2024-03-15T07:01:33Z)
• On Mask-based Image Set Desensitization with Recognition Support [46.51027529020668]
  We propose a mask-based image desensitization approach while supporting recognition. We exploit an interpretation algorithm to maintain critical information for the recognition task. In addition, we propose a feature selection masknet as the model adjustment method to improve the performance based on the masked images.
  arXiv  Detail & Related papers  (2023-12-14T14:26:42Z)
• Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generation [78.13793505707952]
  Existing autoregressive models follow the two-stage generation paradigm that first learns a codebook in the latent space for image reconstruction and then completes the image generation autoregressively based on the learned codebook. We propose a novel two-stage framework, which consists of Masked Quantization VAE (MQ-VAE) Stack model from modeling redundancy.
  arXiv  Detail & Related papers  (2023-05-23T02:15:53Z)
• Improving Masked Autoencoders by Learning Where to Mask [65.89510231743692]
  Masked image modeling is a promising self-supervised learning method for visual data. We present AutoMAE, a framework that uses Gumbel-Softmax to interlink an adversarially-trained mask generator and a mask-guided image modeling process. In our experiments, AutoMAE is shown to provide effective pretraining models on standard self-supervised benchmarks and downstream tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:28:55Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• Exploring the Coordination of Frequency and Attention in Masked Image Modeling [28.418445136155512]
  Masked image modeling (MIM) has dominated self-supervised learning in computer vision. We propose the Frequency & Attention-driven Masking and Throwing Strategy (FAMT), which can extract semantic patches and reduce the number of training patches. FAMT can be seamlessly integrated as a plug-and-play module and surpasses previous works.
  arXiv  Detail & Related papers  (2022-11-28T14:38:19Z)
• Stare at What You See: Masked Image Modeling without Reconstruction [154.74533119863864]
  Masked Autoencoders (MAE) have been prevailing paradigms for large-scale vision representation pre-training. Recent approaches apply semantic-rich teacher models to extract image features as the reconstruction target, leading to better performance. We argue the features extracted by powerful teacher models already encode rich semantic correlation across regions in an intact image.
  arXiv  Detail & Related papers  (2022-11-16T12:48:52Z)
• Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment [157.1292674649519]
  We propose a practical solution named degraded-reference IQA (DR-IQA) DR-IQA exploits the inputs of IR models, degraded images, as references. Our results can even be close to the performance of full-reference settings.
  arXiv  Detail & Related papers  (2021-08-18T02:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.