Multifocal Stereoscopic Projection Mapping

• URL: http://arxiv.org/abs/2110.07726v1
• Date: Fri, 8 Oct 2021 06:13:10 GMT
• Title: Multifocal Stereoscopic Projection Mapping
• Authors: Sorashi Kimura, Daisuke Iwai, Parinya Punpongsanon, Kosuke Sato
• Abstract summary: Current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues. We propose a multifocal approach to mitigate a vergence--accommodation conflict (VAC) in stereoscopic PM. A 3D CG object is projected from a synchronized high-speed projector only when the virtual image of the projected imagery is located at a desired distance.
• Score: 24.101349988126692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stereoscopic projection mapping (PM) allows a user to see a three-dimensional (3D) computer-generated (CG) object floating over physical surfaces of arbitrary shapes around us using projected imagery. However, the current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues, which causes a vergence--accommodation conflict (VAC). Therefore, we propose a multifocal approach to mitigate VAC in stereoscopic PM. Our primary technical contribution is to attach electrically focus-tunable lenses (ETLs) to active shutter glasses to control both vergence and accommodation. Specifically, we apply fast and periodical focal sweeps to the ETLs, which causes the "virtual image'" (as an optical term) of a scene observed through the ETLs to move back and forth during each sweep period. A 3D CG object is projected from a synchronized high-speed projector only when the virtual image of the projected imagery is located at a desired distance. This provides an observer with the correct focus cues required. In this study, we solve three technical issues that are unique to stereoscopic PM: (1) The 3D CG object is displayed on non-planar and even moving surfaces; (2) the physical surfaces need to be shown without the focus modulation; (3) the shutter glasses additionally need to be synchronized with the ETLs and the projector. We also develop a novel compensation technique to deal with the "lens breathing" artifact that varies the retinal size of the virtual image through focal length modulation. Further, using a proof-of-concept prototype, we demonstrate that our technique can present the virtual image of a target 3D CG object at the correct depth. Finally, we validate the advantage provided by our technique by comparing it with conventional stereoscopic PM using a user study on a depth-matching task.

Related papers

• VFMM3D: Releasing the Potential of Image by Vision Foundation Model for Monocular 3D Object Detection [80.62052650370416]
  monocular 3D object detection holds significant importance across various applications, including autonomous driving and robotics. In this paper, we present VFMM3D, an innovative framework that leverages the capabilities of Vision Foundation Models (VFMs) to accurately transform single-view images into LiDAR point cloud representations.
  arXiv  Detail & Related papers  (2024-04-15T03:12:12Z)
• DepthSSC: Depth-Spatial Alignment and Dynamic Voxel Resolution for Monocular 3D Semantic Scene Completion [0.4662017507844857]
  DepthSSC is an advanced method for semantic scene completion solely based on monocular cameras. It mitigates spatial misalignment and distortion issues observed in prior methods. It demonstrates its effectiveness in capturing intricate 3D structural details and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-11-28T01:47:51Z)
• Visually Guided Object Grasping [19.71383212064634]
  We show how to represent a grasp or more generally, an alignment between two solids in 3-D projective space using an uncalibrated stereo rig. We perform an analysis of the performances of the visual servoing algorithm and of the grasping precision that can be expected from this type of approach.
  arXiv  Detail & Related papers  (2023-11-21T15:08:17Z)
• NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localization [80.3424839706698]
  We present NeurOCS, a framework that uses instance masks 3D boxes as input to learn 3D object shapes by means of differentiable rendering. Our approach rests on insights in learning a category-level shape prior directly from real driving scenes. We make critical design choices to learn object coordinates more effectively from an object-centric view.
  arXiv  Detail & Related papers  (2023-05-28T16:18:41Z)
• Decoupling Dynamic Monocular Videos for Dynamic View Synthesis [50.93409250217699]
  We tackle the challenge of dynamic view synthesis from dynamic monocular videos in an unsupervised fashion. Specifically, we decouple the motion of the dynamic objects into object motion and camera motion, respectively regularized by proposed unsupervised surface consistency and patch-based multi-view constraints.
  arXiv  Detail & Related papers  (2023-04-04T11:25:44Z)
• Single-shot ToF sensing with sub-mm precision using conventional CMOS sensors [7.114925332582435]
  We present a novel single-shot interferometric ToF camera targeted for precise 3D measurements of dynamic objects. In contrast to conventional ToF cameras, our device uses only off-the-shelf CCD/CMOS detectors and works at their native chip resolution. We present 3D measurements of small (cm-sized) objects with > 2 Mp point cloud resolution and up to sub-mm depth precision.
  arXiv  Detail & Related papers  (2022-12-02T01:50:36Z)
• Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation [76.58256020932312]
  Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task. We present a self-supervised learning framework for 3D object motion field estimation from monocular videos.
  arXiv  Detail & Related papers  (2021-10-13T16:45:01Z)
• MonoGRNet: A General Framework for Monocular 3D Object Detection [23.59839921644492]
  We propose MonoGRNet for the amodal 3D object detection from a monocular image via geometric reasoning. MonoGRNet decomposes the monocular 3D object detection task into four sub-tasks including 2D object detection, instance-level depth estimation, projected 3D center estimation and local corner regression. Experiments are conducted on KITTI, Cityscapes and MS COCO datasets.
  arXiv  Detail & Related papers  (2021-04-18T10:07:52Z)
• M3DSSD: Monocular 3D Single Stage Object Detector [82.25793227026443]
  We propose a Monocular 3D Single Stage object Detector (M3DSSD) with feature alignment and asymmetric non-local attention. The proposed M3DSSD achieves significantly better performance than the monocular 3D object detection methods on the KITTI dataset.
  arXiv  Detail & Related papers  (2021-03-24T13:09:11Z)
• One Shot 3D Photography [40.83662583097118]
  We present an end-to-end system for creating and viewing 3D photos. Our 3D photos are captured in a single shot and processed directly on a mobile device.
  arXiv  Detail & Related papers  (2020-08-27T17:59:31Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.