Smooth head tracking for virtual reality applications

• URL: http://arxiv.org/abs/2110.14193v1
• Date: Wed, 27 Oct 2021 05:47:21 GMT
• Title: Smooth head tracking for virtual reality applications
• Authors: Abdenour Amamra
• Abstract summary: We propose a new head-tracking solution for human-machine real-time interaction with virtual 3D environments. This solution leverages RGBD data to compute virtual camera pose according to the movements of the user's head.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose a new head-tracking solution for human-machine real-time interaction with virtual 3D environments. This solution leverages RGBD data to compute virtual camera pose according to the movements of the user's head. The process starts with the extraction of a set of facial features from the images delivered by the sensor. Such features are matched against their respective counterparts in a reference image for the computation of the current head pose. Afterwards, a prediction approach is used to guess the most likely next head move (final pose). Pythagorean Hodograph interpolation is then adapted to determine the path and local frames taken between the two poses. The result is a smooth head trajectory that serves as an input to set the camera in virtual scenes according to the user's gaze. The resulting motion model has the advantage of being: continuous in time, it adapts to any frame rate of rendering; it is ergonomic, as it frees the user from wearing tracking markers; it is smooth and free from rendering jerks; and it is also torsion and curvature minimizing as it produces a path with minimum bending energy.

Related papers

• IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera [7.515256982860307]
  IncEventGS is an incremental 3D Gaussian splatting reconstruction algorithm with a single event camera. We exploit the tracking and mapping paradigm of conventional SLAM pipelines for IncEventGS.
  arXiv  Detail & Related papers  (2024-10-10T16:54:23Z)
• 3D Human Scan With A Moving Event Camera [7.734104968315144]
  Event cameras have the advantages of high temporal resolution and high dynamic range. This paper proposes a novel event-based method for 3D pose estimation and human mesh recovery.
  arXiv  Detail & Related papers  (2024-04-12T14:34:24Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Controllable Dynamic Appearance for Neural 3D Portraits [54.29179484318194]
  We propose CoDyNeRF, a system that enables the creation of fully controllable 3D portraits in real-world capture conditions. CoDyNeRF learns to approximate illumination dependent effects via a dynamic appearance model. We demonstrate the effectiveness of our method on free view synthesis of a portrait scene with explicit head pose and expression controls.
  arXiv  Detail & Related papers  (2023-09-20T02:24:40Z)
• FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flow [26.528667940013598]
  Reconstruction of 3D neural fields from posed images has emerged as a promising method for self-supervised representation learning. Key challenge preventing the deployment of these 3D scene learners on large-scale video data is their dependence on precise camera poses from structure-from-motion. We propose a method that jointly reconstructs camera poses and 3D neural scene representations online and in a single forward pass.
  arXiv  Detail & Related papers  (2023-05-31T20:58:46Z)
• Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos [115.71874459429381]
  We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. Experiments on benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.
  arXiv  Detail & Related papers  (2021-11-29T11:25:14Z)
• PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering [56.762094966235566]
  A Portrait Image Neural Renderer is proposed to control the face motions with the parameters of three-dimensional morphable face models. The proposed model can generate photo-realistic portrait images with accurate movements according to intuitive modifications. Our model can generate coherent videos with convincing movements from only a single reference image and a driving audio stream.
  arXiv  Detail & Related papers  (2021-09-17T07:24:16Z)
• Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis [105.37072293076767]
  How to effectively represent camera pose is an essential problem in 3D computer vision. We propose an approach to learn neural representations of camera poses and 3D scenes. We conduct extensive experiments on synthetic and real datasets.
  arXiv  Detail & Related papers  (2021-04-04T00:40:53Z)
• UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality [12.349749717823736]
  In this paper, we propose a new data-driven framework for inside-out body tracking. We first collect a large-scale motion capture dataset with both body and finger motions. We then simulate the occlusion patterns in head-mounted camera views on the captured ground truth using a ray casting algorithm and learn a deep neural network to infer the occluded body parts.
  arXiv  Detail & Related papers  (2020-11-12T09:31:09Z)
• I Like to Move It: 6D Pose Estimation as an Action Decision Process [53.63776807432945]
  Object pose estimation is an integral part of robot vision and AR. Previous 6D pose retrieval pipelines treat the problem either as a regression task or discretize the pose space to classify. We change this paradigm and reformulate the problem as an action decision process where an initial pose is updated in incremental discrete steps. A neural network estimates likely moves from a single RGB image iteratively and determines so an acceptable final pose.
  arXiv  Detail & Related papers  (2020-09-26T20:05:42Z)
• Real-time Simultaneous 3D Head Modeling and Facial Motion Capture with an RGB-D camera [2.3260877354419254]
  We propose a method to build in real-time animated 3D head models using a consumer-grade RGB-D camera. Anyone's head can be instantly reconstructed and his facial motion captured without requiring any training or pre-scanning.
  arXiv  Detail & Related papers  (2020-04-22T13:22:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.