MagicEyes: A Large Scale Eye Gaze Estimation Dataset for Mixed Reality

• URL: http://arxiv.org/abs/2003.08806v1
• Date: Wed, 18 Mar 2020 08:23:57 GMT
• Title: MagicEyes: A Large Scale Eye Gaze Estimation Dataset for Mixed Reality
• Authors: Zhengyang Wu, Srivignesh Rajendran, Tarrence van As, Joelle Zimmermann, Vijay Badrinarayanan, Andrew Rabinovich
• Abstract summary: We present MagicEyes, the first large scale eye dataset collected using real MR devices with comprehensive ground truth labeling. We evaluate several state-of-the-art methods on MagicEyes and also propose a new multi-task EyeNet model designed for detecting the cornea, glints and pupil along with eye segmentation in a single forward pass.
• Score: 8.025086113117291
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the emergence of Virtual and Mixed Reality (XR) devices, eye tracking has received significant attention in the computer vision community. Eye gaze estimation is a crucial component in XR -- enabling energy efficient rendering, multi-focal displays, and effective interaction with content. In head-mounted XR devices, the eyes are imaged off-axis to avoid blocking the field of view. This leads to increased challenges in inferring eye related quantities and simultaneously provides an opportunity to develop accurate and robust learning based approaches. To this end, we present MagicEyes, the first large scale eye dataset collected using real MR devices with comprehensive ground truth labeling. MagicEyes includes $587$ subjects with $80,000$ images of human-labeled ground truth and over $800,000$ images with gaze target labels. We evaluate several state-of-the-art methods on MagicEyes and also propose a new multi-task EyeNet model designed for detecting the cornea, glints and pupil along with eye segmentation in a single forward pass.

Related papers

• EyeSeg: An Uncertainty-Aware Eye Segmentation Framework for AR/VR [58.33693755009173]
  EyeSeg is an uncertainty-aware eye segmentation framework for augmented reality (AR) and virtual reality (VR)<n>We show that EyeSeg achieves segmentation improvements of MIoU, E1, F1, and ACC surpassing previous approaches.
  arXiv  Detail & Related papers  (2025-07-13T14:33:10Z)
• EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Model [51.66031028717933]
  Medical Large Vision-Language Models (Med-LVLMs) demonstrate significant potential in healthcare. Currently, intelligent ophthalmic diagnosis faces three major challenges: (i) Data; (ii) Benchmark; and (iii) Model. We propose the Eyecare Kit, which tackles the aforementioned three key challenges with the tailored dataset, benchmark and model.
  arXiv  Detail & Related papers  (2025-04-18T12:09:15Z)
• The Detection of Saccadic Eye Movements and Per-Eye Comparisons using Virtual Reality Eye Tracking Devices [0.0]
  The study involves VR eye tracking technology and neuroscience with respect to saccadic eye movements. It is anticipated that the software will be able to accurately detect when saccades occur and analyze the differences in saccadic eye movements per-eye.
  arXiv  Detail & Related papers  (2025-03-11T22:15:39Z)
• EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing [2.9795443606634917]
  EyeTrAES is a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement. We show that EyeTrAES boosts pupil tracking fidelity by 6+%, achieving IoU=92%, while incurring at least 3x lower latency than competing pure event-based eye tracking alternatives. For robust user authentication, we train a lightweight per-user Random Forest classifier using a novel feature vector of short-term pupillary kinematics.
  arXiv  Detail & Related papers  (2024-09-27T15:06:05Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions [60.360919642038]
  We propose to predict 3D eye gaze from weak supervision of eye semantic segmentation masks and direct supervision of a few 3D gaze vectors. Our experiments in diverse settings illustrate the significant benefits of the proposed method, achieving about 5 degrees lower angular gaze error over the baseline.
  arXiv  Detail & Related papers  (2023-11-20T20:22:55Z)
• DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery [4.940128337433944]
  We propose a lightweight framework derived from the U-Net 3+ deep learning backbone to estimate measurable periocular depth maps. Our method reconstructs three-dimensional periocular regions, providing a metric basis for related light stimulus calculation protocols and medical guidelines. Evaluated on a sample of 36 participants, our method exhibited notable efficacy in the periocular global precision evaluation experiment, and the pupil diameter measurement.
  arXiv  Detail & Related papers  (2023-11-13T10:55:05Z)
• High-Fidelity Eye Animatable Neural Radiance Fields for Human Face [22.894881396543926]
  We learn a face NeRF model that is sensitive to eye movements from multi-view images. We show that our model is capable of generating high-fidelity images with accurate eyeball rotation and non-rigid periocular deformation.
  arXiv  Detail & Related papers  (2023-08-01T18:26:55Z)
• Near-infrared and visible-light periocular recognition with Gabor features using frequency-adaptive automatic eye detection [69.35569554213679]
  Periocular recognition has gained attention recently due to demands of increased robustness of face or iris in less controlled scenarios. We present a new system for eye detection based on complex symmetry filters, which has the advantage of not needing training. This system is used as input to a periocular algorithm based on retinotopic sampling grids and Gabor spectrum decomposition.
  arXiv  Detail & Related papers  (2022-11-10T13:04:03Z)
• A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications [56.458448869572294]
  We introduce DETRtime, a novel framework for time-series segmentation of EEG data. Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront. Our model generalizes well in the task of EEG sleep stage segmentation.
  arXiv  Detail & Related papers  (2022-06-17T10:17:24Z)
• Gaze Estimation with Eye Region Segmentation and Self-Supervised Multistream Learning [8.422257363944295]
  We present a novel multistream network that learns robust eye representations for gaze estimation. We first create a synthetic dataset containing eye region masks detailing the visible eyeball and iris using a simulator. We then perform eye region segmentation with a U-Net type model which we later use to generate eye region masks for real-world images.
  arXiv  Detail & Related papers  (2021-12-15T04:44:45Z)
• Do Pedestrians Pay Attention? Eye Contact Detection in the Wild [75.54077277681353]
  In urban environments, humans rely on eye contact for fast and efficient communication with nearby people. In this paper, we focus on eye contact detection in the wild, i.e., real-world scenarios for autonomous vehicles with no control over the environment or the distance of pedestrians. We introduce a model that leverages semantic keypoints to detect eye contact and show that this high-level representation achieves state-of-the-art results on the publicly-available dataset JAAD. To study domain adaptation, we create LOOK: a large-scale dataset for eye contact detection in the wild, which focuses on diverse and un
  arXiv  Detail & Related papers  (2021-12-08T10:21:28Z)
• MTCD: Cataract Detection via Near Infrared Eye Images [69.62768493464053]
  cataract is a common eye disease and one of the leading causes of blindness and vision impairment. We present a novel algorithm for cataract detection using near-infrared eye images. Deep learning-based eye segmentation and multitask network classification networks are presented.
  arXiv  Detail & Related papers  (2021-10-06T08:10:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.