PanoTree: Autonomous Photo-Spot Explorer in Virtual Reality Scenes

• URL: http://arxiv.org/abs/2405.17136v2
• Date: Wed, 12 Jun 2024 13:13:10 GMT
• Title: PanoTree: Autonomous Photo-Spot Explorer in Virtual Reality Scenes
• Authors: Tomohiro Hayase, Sacha Braun, Hikari Yanagawa, Itsuki Orito, Yuichi Hiroi,
• Abstract summary: In social VR, photography within a VR scene is an important indicator of visitors' activities. We propose PanoTree, an automated photo-spot explorer in VR scenes. A deep scoring network is trained on a large dataset of photos collected by a social VR platform to determine whether humans are likely to take similar photos.
• Score: 2.4140502941897544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social VR platforms enable social, economic, and creative activities by allowing users to create and share their own virtual spaces. In social VR, photography within a VR scene is an important indicator of visitors' activities. Although automatic identification of photo spots within a VR scene can facilitate the process of creating a VR scene and enhance the visitor experience, there are challenges in quantitatively evaluating photos taken in the VR scene and efficiently exploring the large VR scene. We propose PanoTree, an automated photo-spot explorer in VR scenes. To assess the aesthetics of images captured in VR scenes, a deep scoring network is trained on a large dataset of photos collected by a social VR platform to determine whether humans are likely to take similar photos. Furthermore, we propose a Hierarchical Optimistic Optimization (HOO)-based search algorithm to efficiently explore 3D VR spaces with the reward from the scoring network. Our user study shows that the scoring network achieves human-level performance in distinguishing randomly taken images from those taken by humans. In addition, we show applications using the explored photo spots, such as automatic thumbnail generation, support for VR world creation, and visitor flow planning within a VR scene.

Related papers

• Deep Motion Masking for Secure, Usable, and Scalable Real-Time Anonymization of Virtual Reality Motion Data [49.68609500290361]
  Recent studies have demonstrated that the motion tracking "telemetry" data used by nearly all VR applications is as uniquely identifiable as a fingerprint scan. We present in this paper a state-of-the-art VR identification model that can convincingly bypass known defensive countermeasures.
  arXiv  Detail & Related papers  (2023-11-09T01:34:22Z)
• VR.net: A Real-world Dataset for Virtual Reality Motion Sickness Research [33.092692299254814]
  We introduce VR.net', a dataset offering approximately 12-hour gameplay videos from ten real-world games in 10 diverse genres. For each video frame, a rich set of motion sickness-related labels, such as camera/object movement, depth field, and motion flow, are accurately assigned. We utilize a tool to automatically and precisely extract ground truth data from 3D engines' rendering pipelines without accessing VR games' source code.
  arXiv  Detail & Related papers  (2023-06-06T03:43:11Z)
• Towards a Pipeline for Real-Time Visualization of Faces for VR-based Telepresence and Live Broadcasting Utilizing Neural Rendering [58.720142291102135]
  Head-mounted displays (HMDs) for Virtual Reality pose a considerable obstacle for a realistic face-to-face conversation in VR. We present an approach that focuses on low-cost hardware and can be used on a commodity gaming computer with a single GPU.
  arXiv  Detail & Related papers  (2023-01-04T08:49:51Z)
• Towards 3D VR-Sketch to 3D Shape Retrieval [128.47604316459905]
  We study the use of 3D sketches as an input modality and advocate a VR-scenario where retrieval is conducted. As a first stab at this new 3D VR-sketch to 3D shape retrieval problem, we make four contributions.
  arXiv  Detail & Related papers  (2022-09-20T22:04:31Z)
• Deep Billboards towards Lossless Real2Sim in Virtual Reality [20.7032774699291]
  We develop Deep Billboards that model 3D objects implicitly using neural networks. Our system, connecting a commercial VR headset with a server running neural rendering, allows real-time high-resolution simulation of detailed rigid objects. We augment Deep Billboards with physical interaction capability, adapting classic billboards from screen-based games to immersive VR.
  arXiv  Detail & Related papers  (2022-08-08T16:16:29Z)
• Wireless Edge-Empowered Metaverse: A Learning-Based Incentive Mechanism for Virtual Reality [102.4151387131726]
  We propose a learning-based Incentive Mechanism framework for VR services in the Metaverse. First, we propose the quality of perception as the metric for VR users in the virtual world. Second, for quick trading of VR services between VR users (i.e., buyers) and VR SPs (i.e., sellers), we design a double Dutch auction mechanism. Third, for auction communication reduction, we design a deep reinforcement learning-based auctioneer to accelerate this auction process.
  arXiv  Detail & Related papers  (2021-11-07T13:02:52Z)
• Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Reality [68.18446501943585]
  Social presence will fuel the next generation of communication systems driven by digital humans in virtual reality (VR) The best 3D video-realistic VR avatars that minimize the uncanny effect rely on person-specific (PS) models. This paper makes progress in overcoming these limitations by proposing an end-to-end multi-identity architecture.
  arXiv  Detail & Related papers  (2021-04-10T15:48:53Z)
• Guidelines for the Development of Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: The Development of Virtual Reality Everyday Assessment Lab (VR-EAL) [0.0]
  This study offers guidelines for the development of VR software in cognitive neuroscience and neuropsychology. Twenty-five participants aged between 20 and 45 years with 12-16 years of full-time education evaluated various versions of VR-EAL. The final version of VR-EAL achieved high scores in every sub-score of the VRNQ and exceeded its parsimonious cut-offs.
  arXiv  Detail & Related papers  (2021-01-20T14:55:57Z)
• Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions Without the Presence of Pertinent Adverse Symptomatology [0.0]
  The VRNQ was developed to assess the quality of VR software in terms of user experience, game mechanics, in-game assistance, and VRISE. The maximum duration of VR sessions should be between 55-70 minutes when the VR software meets or exceeds the parsimonious cut-offs of the VRNQ. Deeper immersion, better quality of graphics and sound, and more helpful in-game instructions and prompts were found to reduce VRISE intensity.
  arXiv  Detail & Related papers  (2021-01-20T14:10:44Z)
• Unmasking Communication Partners: A Low-Cost AI Solution for Digitally Removing Head-Mounted Displays in VR-Based Telepresence [62.997667081978825]
  Face-to-face conversation in Virtual Reality (VR) is a challenge when participants wear head-mounted displays (HMD) Past research has shown that high-fidelity face reconstruction with personal avatars in VR is possible under laboratory conditions with high-cost hardware. We propose one of the first low-cost systems for this task which uses only open source, free software and affordable hardware.
  arXiv  Detail & Related papers  (2020-11-06T23:17:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.