DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery
- URL: http://arxiv.org/abs/2311.07235v1
- Date: Mon, 13 Nov 2023 10:55:05 GMT
- Title: DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery
- Authors: Yitong Sun, Zijian Zhou, Cyriel Diels, Ali Asadipour
- Abstract summary: We propose a lightweight framework derived from the U-Net 3+ deep learning backbone to estimate measurable periocular depth maps.
Our method reconstructs three-dimensional periocular regions, providing a metric basis for related light stimulus calculation protocols and medical guidelines.
Evaluated on a sample of 36 participants, our method exhibited notable efficacy in the periocular global precision evaluation experiment, and the pupil diameter measurement.
- Score: 4.940128337433944
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Despite the enhanced realism and immersion provided by VR headsets, users
frequently encounter adverse effects such as digital eye strain (DES), dry eye,
and potential long-term visual impairment due to excessive eye stimulation from
VR displays and pressure from the mask. Recent VR headsets are increasingly
equipped with eye-oriented monocular cameras to segment ocular feature maps.
Yet, to compute the incident light stimulus and observe periocular condition
alterations, it is imperative to transform these relative measurements into
metric dimensions. To bridge this gap, we propose a lightweight framework
derived from the U-Net 3+ deep learning backbone that we re-optimised, to
estimate measurable periocular depth maps. Compatible with any VR headset
equipped with an eye-oriented monocular camera, our method reconstructs
three-dimensional periocular regions, providing a metric basis for related
light stimulus calculation protocols and medical guidelines. Navigating the
complexities of data collection, we introduce a Dynamic Periocular Data
Generation (DPDG) environment based on UE MetaHuman, which synthesises
thousands of training images from a small quantity of human facial scan data.
Evaluated on a sample of 36 participants, our method exhibited notable efficacy
in the periocular global precision evaluation experiment, and the pupil
diameter measurement.
Related papers
- Enhanced Scale-aware Depth Estimation for Monocular Endoscopic Scenes with Geometric Modeling [42.70053750500301]
We propose a novel scale-aware framework that only uses monocular images with geometric modeling for depth estimation.
Specifically, we first propose a multi-resolution depth fusion strategy to enhance the quality of monocular depth estimation.
By coupling scale factors and relative depth estimation, the scale-aware depth of the monocular endoscopic scenes can be estimated.
arXiv Detail & Related papers (2024-08-14T03:18:04Z) - High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces [18.948630080040576]
We introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map.
Our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface.
This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
arXiv Detail & Related papers (2024-04-20T18:06:26Z) - Accurate Eye Tracking from Dense 3D Surface Reconstructions using
Single-Shot Deflectometry [14.26583534657278]
We propose a novel method for accurate and fast evaluation of the gaze direction that exploits teachings from single-shot phase-measuring-deflectometry (PMD)
Our method acquires dense 3D surface information of both cornea and sclera within only one single camera frame (single-shot)
arXiv Detail & Related papers (2023-08-14T17:36:39Z) - Optimization-Based Eye Tracking using Deflectometric Information [14.010352335803873]
State-of-the-art eye tracking methods are either-based and track reflections of sparse point light sources, or image-based and exploit 2D features of the acquired eye image.
We develop a differentiable pipeline based on PyTorch3D that simulates a virtual eye under screen illumination.
In general, our method does not require a specific pattern rendering and can work with ordinary video frames of the main VR/AR/MR screen itself.
arXiv Detail & Related papers (2023-03-09T02:41:13Z) - A Deep Learning Approach for the Segmentation of Electroencephalography
Data in Eye Tracking Applications [56.458448869572294]
We introduce DETRtime, a novel framework for time-series segmentation of EEG data.
Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront.
Our model generalizes well in the task of EEG sleep stage segmentation.
arXiv Detail & Related papers (2022-06-17T10:17:24Z) - Depth Estimation Matters Most: Improving Per-Object Depth Estimation for
Monocular 3D Detection and Tracking [47.59619420444781]
Approaches to monocular 3D perception including detection and tracking often yield inferior performance when compared to LiDAR-based techniques.
We propose a multi-level fusion method that combines different representations (RGB and pseudo-LiDAR) and temporal information across multiple frames for objects (tracklets) to enhance per-object depth estimation.
arXiv Detail & Related papers (2022-06-08T03:37:59Z) - 3D endoscopic depth estimation using 3D surface-aware constraints [16.161276518580262]
We show that depth estimation can be reformed from a 3D surface perspective.
We propose a loss function for depth estimation that integrates the surface-aware constraints.
Camera parameters are incorporated into the training pipeline to increase the control and transparency of the depth estimation.
arXiv Detail & Related papers (2022-03-04T04:47:20Z) - Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
We present a modern solution to the multi-view photometric stereo problem (MVPS)
We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry.
Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
arXiv Detail & Related papers (2021-10-11T20:20:03Z) - SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera [97.0162841635425]
We present a solution to egocentric 3D body pose estimation from monocular images captured from downward looking fish-eye cameras installed on the rim of a head mounted VR device.
This unusual viewpoint leads to images with unique visual appearance, with severe self-occlusions and perspective distortions.
We propose an encoder-decoder architecture with a novel multi-branch decoder designed to account for the varying uncertainty in 2D predictions.
arXiv Detail & Related papers (2020-11-02T16:18:06Z) - Single View Metrology in the Wild [94.7005246862618]
We present a novel approach to single view metrology that can recover the absolute scale of a scene represented by 3D heights of objects or camera height above the ground.
Our method relies on data-driven priors learned by a deep network specifically designed to imbibe weakly supervised constraints from the interplay of the unknown camera with 3D entities such as object heights.
We demonstrate state-of-the-art qualitative and quantitative results on several datasets as well as applications including virtual object insertion.
arXiv Detail & Related papers (2020-07-18T22:31:33Z) - D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual
Odometry [57.5549733585324]
D3VO is a novel framework for monocular visual odometry that exploits deep networks on three levels -- deep depth, pose and uncertainty estimation.
We first propose a novel self-supervised monocular depth estimation network trained on stereo videos without any external supervision.
We model the photometric uncertainties of pixels on the input images, which improves the depth estimation accuracy.
arXiv Detail & Related papers (2020-03-02T17:47:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.