A deep learning approach to track eye movements based on events

• URL: http://arxiv.org/abs/2508.04827v1
• Date: Wed, 06 Aug 2025 19:12:42 GMT
• Title: A deep learning approach to track eye movements based on events
• Authors: Chirag Seth, Divya Naiken, Keyan Lin,
• Abstract summary: This research project addresses the challenge of accurately tracking eye movements during specific events by leveraging previous research.<n>Our primary objective is to locate the eye center position (x, y) using inputs from an event camera.<n>Our ultimate goal is to develop an interpretable and cost-effective algorithm using deep learning methods to predict human attention.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This research project addresses the challenge of accurately tracking eye movements during specific events by leveraging previous research. Given the rapid movements of human eyes, which can reach speeds of 300{\deg}/s, precise eye tracking typically requires expensive and high-speed cameras. Our primary objective is to locate the eye center position (x, y) using inputs from an event camera. Eye movement analysis has extensive applications in consumer electronics, especially in VR and AR product development. Therefore, our ultimate goal is to develop an interpretable and cost-effective algorithm using deep learning methods to predict human attention, thereby improving device comfort and enhancing overall user experience. To achieve this goal, we explored various approaches, with the CNN\_LSTM model proving most effective, achieving approximately 81\% accuracy. Additionally, we propose future work focusing on Layer-wise Relevance Propagation (LRP) to further enhance the model's interpretability and predictive performance.

Related papers

• Exploring Temporal Dynamics in Event-based Eye Tracker [3.3325719644030016]
  Eye-tracking is a vital technology for human-computer interaction, especially in wearable devices such as AR, VR, and XR.<n>The realization of high-speed and high-precision eye-tracking using frame-based image sensors is constrained by their limited temporal resolution.<n>We propose TDTracker, an effective eye-tracking framework that captures rapid eye movements by thoroughly modeling temporal dynamics.
  arXiv  Detail & Related papers  (2025-03-31T04:57:13Z)
• A Cross-Scene Benchmark for Open-World Drone Active Tracking [54.235808061746525]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT.<n>We also propose a reinforcement learning-based drone tracking method called R-VAT.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• E-Motion: Future Motion Simulation via Event Sequence Diffusion [86.80533612211502]
  Event-based sensors may potentially offer a unique opportunity to predict future motion with a level of detail and precision previously unachievable. We propose to integrate the strong learning capacity of the video diffusion model with the rich motion information of an event camera as a motion simulation framework. Our findings suggest a promising direction for future research in enhancing the interpretative power and predictive accuracy of computer vision systems.
  arXiv  Detail & Related papers  (2024-10-11T09:19:23Z)
• A Framework for Pupil Tracking with Event Cameras [1.708806485130162]
  Saccades are extremely rapid movements of both eyes that occur simultaneously. The peak angular speed of the eye during a saccade can reach as high as 700deg/s in humans. We present events as frames that can be readily utilized by standard deep learning algorithms.
  arXiv  Detail & Related papers  (2024-07-23T17:32:02Z)
• Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality [2.2639735235640015]
  This work provides an objective assessment of the impact of several contemporary machine learning (ML)-based methods for eye feature tracking. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.
  arXiv  Detail & Related papers  (2024-03-28T18:43:25Z)
• Open Gaze: Open Source eye tracker for smartphone devices using Deep Learning [0.0]
  We present an open-source implementation of a smartphone-based gaze tracker that emulates the methodology proposed by a GooglePaper. Through the integration of machine learning techniques, we unveil an accurate eye tracking solution that is native to smartphones. Our findings exhibit the inherent potential to amplify eye movement research by significant proportions.
  arXiv  Detail & Related papers  (2023-08-25T17:10:22Z)
• A Real-time Human Pose Estimation Approach for Optimal Sensor Placement in Sensor-based Human Activity Recognition [63.26015736148707]
  This paper introduces a novel methodology to resolve the issue of optimal sensor placement for Human Activity Recognition. The derived skeleton data provides a unique strategy for identifying the optimal sensor location. Our findings indicate that the vision-based method for sensor placement offers comparable results to the conventional deep learning approach.
  arXiv  Detail & Related papers  (2023-07-06T10:38:14Z)
• CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild [18.79132232751083]
  Real-time analysis of the dynamics of the eye region allows us to monitor humans' visual attention allocation and estimate their mental state. We propose CLERA, which achieves precise keypoint detection andtemporal tracking in a joint-learning framework. We also introduce a large-scale dataset of 30k human faces with joint pupil, eye-openness, and landmark annotation.
  arXiv  Detail & Related papers  (2023-06-26T21:20:23Z)
• Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks [55.81577205593956]
  Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously. Deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential.
  arXiv  Detail & Related papers  (2023-02-17T14:19:28Z)
• A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications [56.458448869572294]
  We introduce DETRtime, a novel framework for time-series segmentation of EEG data. Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront. Our model generalizes well in the task of EEG sleep stage segmentation.
  arXiv  Detail & Related papers  (2022-06-17T10:17:24Z)
• GIMO: Gaze-Informed Human Motion Prediction in Context [75.52839760700833]
  We propose a large-scale human motion dataset that delivers high-quality body pose sequences, scene scans, and ego-centric views with eye gaze. Our data collection is not tied to specific scenes, which further boosts the motion dynamics observed from our subjects. To realize the full potential of gaze, we propose a novel network architecture that enables bidirectional communication between the gaze and motion branches.
  arXiv  Detail & Related papers  (2022-04-20T13:17:39Z)
• Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations [75.60524561611008]
  This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
  arXiv  Detail & Related papers  (2021-09-28T20:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.