Enhancing Content Representation for AR Image Quality Assessment Using Knowledge Distillation

• URL: http://arxiv.org/abs/2412.06003v1
• Date: Sun, 08 Dec 2024 17:25:30 GMT
• Title: Enhancing Content Representation for AR Image Quality Assessment Using Knowledge Distillation
• Authors: Aymen Sekhri, Seyed Ali Amirshahi, Mohamed-Chaker Larabi,
• Abstract summary: This paper presents a deep learning-based objective metric designed specifically for assessing image quality for Augmented Reality scenarios.<n>It entails four key steps, (1) fine-tuning a self-supervised pre-trained vision transformer to extract prominent features from reference images, (2) quantifying distortions by computing shift representations, (3) employing cross-attention-based decoders to capture perceptual quality features, and (4) integrating regularization techniques and label smoothing to address the overfitting problem.
• Score: 3.020452010930984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Augmented Reality (AR) is a major immersive media technology that enriches our perception of reality by overlaying digital content (the foreground) onto physical environments (the background). It has far-reaching applications, from entertainment and gaming to education, healthcare, and industrial training. Nevertheless, challenges such as visual confusion and classical distortions can result in user discomfort when using the technology. Evaluating AR quality of experience becomes essential to measure user satisfaction and engagement, facilitating the refinement necessary for creating immersive and robust experiences. Though, the scarcity of data and the distinctive characteristics of AR technology render the development of effective quality assessment metrics challenging. This paper presents a deep learning-based objective metric designed specifically for assessing image quality for AR scenarios. The approach entails four key steps, (1) fine-tuning a self-supervised pre-trained vision transformer to extract prominent features from reference images and distilling this knowledge to improve representations of distorted images, (2) quantifying distortions by computing shift representations, (3) employing cross-attention-based decoders to capture perceptual quality features, and (4) integrating regularization techniques and label smoothing to address the overfitting problem. To validate the proposed approach, we conduct extensive experiments on the ARIQA dataset. The results showcase the superior performance of our proposed approach across all model variants, namely TransformAR, TransformAR-KD, and TransformAR-KD+ in comparison to existing state-of-the-art methods.

Related papers

• Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images [70.49595920462579]
  This work studies the quality assessment and distortion-aware saliency prediction problems for AIGODIs.<n>We propose two models with shared encoders based on the BLIP-2 model to evaluate the human visual experience and predict distortion-aware saliency for AI-generated omnidirectional images.
  arXiv  Detail & Related papers  (2025-06-27T05:36:04Z)
• AI-powered Contextual 3D Environment Generation: A Systematic Review [49.1574468325115]
  This study performs a systematic review of existing generative AI techniques for 3D scene generation.<n>By examining state-of-the-art approaches, it presents key challenges such as scene authenticity and the influence of textual inputs.
  arXiv  Detail & Related papers  (2025-06-05T15:56:28Z)
• DGIQA: Depth-guided Feature Attention and Refinement for Generalizable Image Quality Assessment [9.851063768646847]
  A long-held challenge in no-reference image quality assessment is the lack of objective generalization to unseen natural distortions.<n>We integrate a novel Depth-Guided cross-attention and refinement mechanism, which distills scene depth and spatial features into a structure-aware representation.<n>We implement TCB and Depth-CAR as multimodal attention-based projection functions to select the most informative features.<n> Experimental results demonstrate that our proposed DGIQA model achieves state-of-the-art (SOTA) performance on both synthetic and authentic benchmark datasets.
  arXiv  Detail & Related papers  (2025-05-29T20:52:56Z)
• Scene Perceived Image Perceptual Score (SPIPS): combining global and local perception for image quality assessment [0.0]
  We propose a novel IQA approach that bridges the gap between deep learning methods and human perception. Our model disentangles deep features into high-level semantic information and low-level perceptual details, treating each stream separately. This hybrid design enables the model to assess both global context and intricate image details, better reflecting the human visual process.
  arXiv  Detail & Related papers  (2025-04-24T04:06:07Z)
• PIGUIQA: A Physical Imaging Guided Perceptual Framework for Underwater Image Quality Assessment [59.9103803198087]
  We propose a Physical Imaging Guided perceptual framework for Underwater Image Quality Assessment (UIQA) By leveraging underwater radiative transfer theory, we integrate physics-based imaging estimations to establish quantitative metrics for these distortions. The proposed model accurately predicts image quality scores and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-20T03:31:45Z)
• A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
  Image restoration (IR) refers to the process of improving visual quality of images while removing degradation, such as noise, blur, weather effects, and so on. Traditional IR methods typically target specific types of degradation, which limits their effectiveness in real-world scenarios with complex distortions. The all-in-one image restoration (AiOIR) paradigm has emerged, offering a unified framework that adeptly addresses multiple degradation types.
  arXiv  Detail & Related papers  (2024-10-19T11:11:09Z)
• DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformer [23.70791030264281]
  Generic Face Image Quality Assessment (GFIQA) evaluates the perceptual quality of facial images. We present a novel transformer-based method for GFIQA, which is aided by two unique mechanisms.
  arXiv  Detail & Related papers  (2024-06-13T23:11:25Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• OCR is All you need: Importing Multi-Modality into Image-based Defect Detection System [7.1083241462091165]
  We introduce an external modality-guided data mining framework, primarily rooted in optical character recognition (OCR), to extract statistical features from images. A key aspect of our approach is the alignment of external modality features, extracted using a single modality-aware model, with image features encoded by a convolutional neural network. Our methodology considerably boosts the recall rate of the defect detection model and maintains high robustness even in challenging scenarios.
  arXiv  Detail & Related papers  (2024-03-18T07:41:39Z)
• Fiducial Focus Augmentation for Facial Landmark Detection [4.433764381081446]
  We propose a novel image augmentation technique to enhance the model's understanding of facial structures. We employ a Siamese architecture-based training mechanism with a Deep Canonical Correlation Analysis (DCCA)-based loss. Our approach outperforms multiple state-of-the-art approaches across various benchmark datasets.
  arXiv  Detail & Related papers  (2024-02-23T01:34:00Z)
• Learning from Multi-Perception Features for Real-Word Image Super-resolution [87.71135803794519]
  We propose a novel SR method called MPF-Net that leverages multiple perceptual features of input images. Our method incorporates a Multi-Perception Feature Extraction (MPFE) module to extract diverse perceptual information. We also introduce a contrastive regularization term (CR) that improves the model's learning capability.
  arXiv  Detail & Related papers  (2023-05-26T07:35:49Z)
• Helping Visually Impaired People Take Better Quality Pictures [52.03016269364854]
  We develop tools to help visually impaired users minimize occurrences of common technical distortions. We also create a prototype feedback system that helps to guide users to mitigate quality issues.
  arXiv  Detail & Related papers  (2023-05-14T04:37:53Z)
• MSTRIQ: No Reference Image Quality Assessment Based on Swin Transformer with Multi-Stage Fusion [8.338999282303755]
  We propose a novel algorithm based on the Swin Transformer. It aggregates information from both local and global features to better predict the quality. It ranks 2nd in the no-reference track of NTIRE 2022 Perceptual Image Quality Assessment Challenge.
  arXiv  Detail & Related papers  (2022-05-20T11:34:35Z)
• Confusing Image Quality Assessment: Towards Better Augmented Reality Experience [96.29124666702566]
  We consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. An objective metric termed CFIQA is also proposed to better evaluate the confusing image quality.
  arXiv  Detail & Related papers  (2022-04-11T07:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.