Perceptual Quality Assessment of Omnidirectional Images as Moving Camera Videos

• URL: http://arxiv.org/abs/2005.10547v2
• Date: Tue, 5 Jan 2021 03:23:16 GMT
• Title: Perceptual Quality Assessment of Omnidirectional Images as Moving Camera Videos
• Authors: Xiangjie Sui, Kede Ma, Yiru Yao, Yuming Fang
• Abstract summary: Two types of VR viewing conditions are crucial in determining the viewing behaviors of users and the perceived quality of the panorama. We first transform an omnidirectional image to several video representations using different user viewing behaviors under different viewing conditions. We then leverage advanced 2D full-reference video quality models to compute the perceived quality.
• Score: 49.217528156417906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Omnidirectional images (also referred to as static 360{\deg} panoramas) impose viewing conditions much different from those of regular 2D images. How do humans perceive image distortions in immersive virtual reality (VR) environments is an important problem which receives less attention. We argue that, apart from the distorted panorama itself, two types of VR viewing conditions are crucial in determining the viewing behaviors of users and the perceived quality of the panorama: the starting point and the exploration time. We first carry out a psychophysical experiment to investigate the interplay among the VR viewing conditions, the user viewing behaviors, and the perceived quality of 360{\deg} images. Then, we provide a thorough analysis of the collected human data, leading to several interesting findings. Moreover, we propose a computational framework for objective quality assessment of 360{\deg} images, embodying viewing conditions and behaviors in a delightful way. Specifically, we first transform an omnidirectional image to several video representations using different user viewing behaviors under different viewing conditions. We then leverage advanced 2D full-reference video quality models to compute the perceived quality. We construct a set of specific quality measures within the proposed framework, and demonstrate their promises on three VR quality databases.

Related papers

• Perceptual Depth Quality Assessment of Stereoscopic Omnidirectional Images [10.382801621282228]
  We develop an objective quality assessment model named depth quality index (DQI) for efficient no-reference (NR) depth quality assessment of stereoscopic omnidirectional images. Motivated by the perceptual characteristics of the human visual system (HVS), the proposed DQI is built upon multi-color-channel, adaptive viewport selection, and interocular discrepancy features.
  arXiv  Detail & Related papers  (2024-08-19T16:28:05Z)
• Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting [33.95741744421632]
  We propose a transformer-based 360 image outpainting framework called Dream360. It can generate diverse, high-fidelity, and high-resolution panoramas from user-selected viewports. Our Dream360 achieves significantly lower Frechet Inception Distance (FID) scores and better visual fidelity than existing methods.
  arXiv  Detail & Related papers  (2024-01-19T09:01:20Z)
• Perceptual Quality Assessment of 360$^\circ$ Images Based on Generative Scanpath Representation [40.00063797833765]
  We introduce a unique generative scanpath representation (GSR) for effective quality inference of 360$circ$ images. GSR aggregates varied perceptual experiences of multi-hypothesis users under a predefined viewing condition. We then propose an efficient OIQA computational framework by learning the quality maps GSR.
  arXiv  Detail & Related papers  (2023-09-07T04:10:30Z)
• State of the Art in Dense Monocular Non-Rigid 3D Reconstruction [100.9586977875698]
  3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views.
  arXiv  Detail & Related papers  (2022-10-27T17:59:53Z)
• Perceptual Quality Assessment of Omnidirectional Images [81.76416696753947]
  We first establish an omnidirectional IQA (OIQA) database, which includes 16 source images and 320 distorted images degraded by 4 commonly encountered distortion types. Then a subjective quality evaluation study is conducted on the OIQA database in the VR environment. The original and distorted omnidirectional images, subjective quality ratings, and the head and eye movement data together constitute the OIQA database.
  arXiv  Detail & Related papers  (2022-07-06T13:40:38Z)
• Perceptual Quality Assessment of Virtual Reality Videos in the Wild [53.94620993606658]
  Existing panoramic video databases only consider synthetic distortions, assume fixed viewing conditions, and are limited in size. We construct the VR Video Quality in the Wild (VRVQW) database, containing $502$ user-generated videos with diverse content and distortion characteristics. We conduct a formal psychophysical experiment to record the scanpaths and perceived quality scores from $139$ participants under two different viewing conditions.
  arXiv  Detail & Related papers  (2022-06-13T02:22:57Z)
• Blind VQA on 360{\deg} Video via Progressively Learning from Pixels, Frames and Video [66.57045901742922]
  Blind visual quality assessment (BVQA) on 360textdegree video plays a key role in optimizing immersive multimedia systems. In this paper, we take into account the progressive paradigm of human perception towards spherical video quality. We propose a novel BVQA approach (namely ProVQA) for 360textdegree video via progressively learning from pixels, frames and video.
  arXiv  Detail & Related papers  (2021-11-18T03:45:13Z)
• ATSal: An Attention Based Architecture for Saliency Prediction in 360 Videos [5.831115928056554]
  This paper proposes ATSal, a novel attention based (head-eye) saliency model for 360degree videos. We compare the proposed approach to other state-of-the-art saliency models on two datasets: Salient360! and VR-EyeTracking. Experimental results on over 80 ODV videos (75K+ frames) show that the proposed method outperforms the existing state-of-the-art.
  arXiv  Detail & Related papers  (2020-11-20T19:19:48Z)
• SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera [97.0162841635425]
  We present a solution to egocentric 3D body pose estimation from monocular images captured from downward looking fish-eye cameras installed on the rim of a head mounted VR device. This unusual viewpoint leads to images with unique visual appearance, with severe self-occlusions and perspective distortions. We propose an encoder-decoder architecture with a novel multi-branch decoder designed to account for the varying uncertainty in 2D predictions.
  arXiv  Detail & Related papers  (2020-11-02T16:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.