Towards Top-Down Stereo Image Quality Assessment via Stereo Attention

• URL: http://arxiv.org/abs/2308.04156v3
• Date: Tue, 14 Nov 2023 09:43:45 GMT
• Title: Towards Top-Down Stereo Image Quality Assessment via Stereo Attention
• Abstract summary: Stereo image quality assessment (SIQA) plays a crucial role in evaluating and improving the visual experience of 3D content. This paper presents a novel Stereo AttenTion Network (SATNet) which employs a top-down perspective to guide the quality assessment process. Experimental results highlight the superiority of our top-down method in advancing the state-of-the-art in the SIQA field.
• Score: 3.113304966059062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stereo image quality assessment (SIQA) plays a crucial role in evaluating and improving the visual experience of 3D content. Existing visual properties-based methods for SIQA have achieved promising performance. However, these approaches ignore the top-down philosophy, leading to a lack of a comprehensive grasp of the human visual system (HVS) and SIQA. This paper presents a novel Stereo AttenTion Network (SATNet), which employs a top-down perspective to guide the quality assessment process. Specifically, our generalized Stereo AttenTion (SAT) structure adapts components and input/output for stereo scenarios. It leverages the fusion-generated attention map as a higher-level binocular modulator to influence two lower-level monocular features, allowing progressive recalibration of both throughout the pipeline. Additionally, we introduce an Energy Coefficient (EC) to flexibly tune the magnitude of binocular response, accounting for the fact that binocular responses in the primate primary visual cortex are less than the sum of monocular responses. To extract the most discriminative quality information from the summation and subtraction of the two branches of monocular features, we utilize a dual-pooling strategy that applies min-pooling and max-pooling operations to the respective branches. Experimental results highlight the superiority of our top-down method in advancing the state-of-the-art in the SIQA field. The code is available at https://github.com/Fanning-Zhang/SATNet.

Related papers

• Blind Stereoscopic Omnidirectional Image Quality Assessment Using Predictive Coding Hierarchy [28.575866509102124]
  We propose a Predictive Coding Hierarchy-inspired metric (PCH) for blind/no-reference stereoscopic omnidirectional image quality assessment.<n>Motivated by the viewing process of SOIs, the proposed PCH includes a local cyclopean perception module, a global predictive perception module, and a visual quality regressor.<n>Extensive experiments demonstrate that the proposed PCH achieves competitive and consistently improved performance compared with state-of-the-art quality assessment methods.
  arXiv  Detail & Related papers  (2026-08-28T19:06:04Z)
• Revisiting Vision Language Foundations for No-Reference Image Quality Assessment [31.550239698285058]
  Large-scale vision language pre-training has recently shown promise for no-reference image-quality assessment (NR-IQA)<n>We present the first systematic evaluation of six prominent pretrained backbones, CLIP, SigLIP2, DINOv2, DINOv3, Perception, and ResNet, for the task of No-Reference Image Quality Assessment (NR-IQA)<n>Our study uncovers two previously overlooked factors: (1) SigLIP2 consistently achieves strong performance; and (2) the choice of activation function plays a surprisingly crucial role, particularly for enhancing the generalization ability of image quality assessment models.
  arXiv  Detail & Related papers  (2025-09-22T06:24:42Z)
• Perception-oriented Bidirectional Attention Network for Image Super-resolution Quality Assessment [61.27648290203618]
  We propose the Perception-oriented Bidirectional Attention Network (PBAN) for image SR FR-IQA.<n>PBAN is composed of three modules: an image encoder module, a perception-oriented bidirectional attention (PBA) module, and a quality prediction module.<n>Our proposed PBAN outperforms state-of-the-art quality assessment methods.
  arXiv  Detail & Related papers  (2025-09-08T08:39:45Z)
• EyeSim-VQA: A Free-Energy-Guided Eye Simulation Framework for Video Quality Assessment [68.77813885751308]
  EyeSimVQA is a novel VQA framework that incorporates free-energy-based self-repair.<n>We show EyeSimVQA achieves competitive or superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-06-13T08:00:54Z)
• DGIQA: Depth-guided Feature Attention and Refinement for Generalizable Image Quality Assessment [9.851063768646847]
  A long-held challenge in no-reference image quality assessment is the lack of objective generalization to unseen natural distortions.<n>We integrate a novel Depth-Guided cross-attention and refinement mechanism, which distills scene depth and spatial features into a structure-aware representation.<n>We implement TCB and Depth-CAR as multimodal attention-based projection functions to select the most informative features.<n> Experimental results demonstrate that our proposed DGIQA model achieves state-of-the-art (SOTA) performance on both synthetic and authentic benchmark datasets.
  arXiv  Detail & Related papers  (2025-05-29T20:52:56Z)
• Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion [88.67015254278859]
  We introduce the Mono2Stereo dataset, providing high-quality training data and benchmark to support in-depth exploration of stereo conversion. We conduct an empirical study that yields two primary findings. 1) The differences between the left and right views are subtle, yet existing metrics consider overall pixels, failing to concentrate on regions critical to stereo effects. We introduce a new evaluation metric, Stereo Intersection-over-Union, which harmonizes disparity and achieves a high correlation with human judgments on stereo effect.
  arXiv  Detail & Related papers  (2025-03-28T09:25:58Z)
• ESVQA: Perceptual Quality Assessment of Egocentric Spatial Videos [71.62145804686062]
  Assessing the quality of experience (QoE) of egocentric spatial videos is crucial to ensure a high-quality viewing experience.<n>We introduce the first Egocentric Spatial Video Quality Assessment Database (ESVQAD), which comprises 600 egocentric spatial videos.<n>We also propose a novel multi-dimensional binocular feature fusion model, termed ESVQAnet, which integrates binocular spatial, motion, and semantic features to predict the overall perceptual quality.
  arXiv  Detail & Related papers  (2024-12-29T10:13:30Z)
• Perceptual Depth Quality Assessment of Stereoscopic Omnidirectional Images [10.382801621282228]
  We develop an objective quality assessment model named depth quality index (DQI) for efficient no-reference (NR) depth quality assessment of stereoscopic omnidirectional images. Motivated by the perceptual characteristics of the human visual system (HVS), the proposed DQI is built upon multi-color-channel, adaptive viewport selection, and interocular discrepancy features.
  arXiv  Detail & Related papers  (2024-08-19T16:28:05Z)
• MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition [62.89464258519723]
  We propose a multi-layer cross-attention fusion based AVSR approach that promotes representation of each modality by fusing them at different levels of audio/visual encoders. Our proposed approach surpasses the first-place system, establishing a new SOTA cpCER of 29.13% on this dataset.
  arXiv  Detail & Related papers  (2024-01-07T08:59:32Z)
• Self-Supervised Monocular Depth Estimation by Direction-aware Cumulative Convolution Network [80.19054069988559]
  We find that self-supervised monocular depth estimation shows a direction sensitivity and environmental dependency. We propose a new Direction-aware Cumulative Convolution Network (DaCCN), which improves the depth representation in two aspects. Experiments show that our method achieves significant improvements on three widely used benchmarks.
  arXiv  Detail & Related papers  (2023-08-10T14:32:18Z)
• Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment [50.82681686110528]
  Blind Omnidirectional Image Quality Assessment (BOIQA) aims to objectively assess the human perceptual quality of omnidirectional images (ODIs) The quality assessment of ODIs is severely hampered by the fact that the existing BOIQA pipeline lacks the modeling of the observer's browsing process. We propose a novel multi-sequence network for BOIQA called Assessor360, which is derived from the realistic multi-assessor ODI quality assessment procedure.
  arXiv  Detail & Related papers  (2023-05-18T13:55:28Z)
• Top-Down Visual Attention from Analysis by Synthesis [87.47527557366593]
  We consider top-down attention from a classic Analysis-by-Synthesis (AbS) perspective of vision. We propose Analysis-by-Synthesis Vision Transformer (AbSViT), which is a top-down modulated ViT model that variationally approximates AbS, and controllable achieves top-down attention.
  arXiv  Detail & Related papers  (2023-03-23T05:17:05Z)
• End-to-end deep multi-score model for No-reference stereoscopic image quality assessment [6.254148286968409]
  We use a deep multi-score Convolutional Neural Network (CNN) to estimate stereoscopic image quality without reference. Our model has been trained to perform four tasks: First, predict the left view's quality. Second, predict the quality of the left view. Third and fourth, predict the quality of the stereo view and global quality, respectively, with the global score serving as the ultimate quality.
  arXiv  Detail & Related papers  (2022-11-02T16:45:35Z)
• Perception-Oriented Stereo Image Super-Resolution [19.226330884091645]
  This paper proposes the first perception-oriented stereo image super-resolution approach by exploiting the feedback, provided by the evaluation on the perceptual quality of StereoSR results. Experiments demonstrate that our StereoSR approach significantly improves the perceptual quality and enhances the reliability of stereo images for disparity estimation.
  arXiv  Detail & Related papers  (2022-07-14T02:13:42Z)
• ChiTransformer:Towards Reliable Stereo from Cues [10.756828396434033]
  Current stereo matching techniques are challenged by restricted searching space, occluded regions and sheer size. We present an optic-chiasm-inspired self-supervised binocular depth estimation method. ChiTransformer architecture yields substantial improvements over state-of-the-art self-supervised stereo approaches by 11%.
  arXiv  Detail & Related papers  (2022-03-09T07:19:58Z)
• SGM3D: Stereo Guided Monocular 3D Object Detection [62.11858392862551]
  We propose a stereo-guided monocular 3D object detection network, termed SGM3D. We exploit robust 3D features extracted from stereo images to enhance the features learned from the monocular image. Our method can be integrated into many other monocular approaches to boost performance without introducing any extra computational cost.
  arXiv  Detail & Related papers  (2021-12-03T13:57:14Z)
• Efficient Self-supervised Vision Transformers for Representation Learning [86.57557009109411]
  We show that multi-stage architectures with sparse self-attentions can significantly reduce modeling complexity. We propose a new pre-training task of region matching which allows the model to capture fine-grained region dependencies. Our results show that combining the two techniques, EsViT achieves 81.3% top-1 on the ImageNet linear probe evaluation.
  arXiv  Detail & Related papers  (2021-06-17T19:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.