EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing

• URL: http://arxiv.org/abs/2409.18813v1
• Date: Fri, 27 Sep 2024 15:06:05 GMT
• Title: EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing
• Authors: Argha Sen, Nuwan Bandara, Ila Gokarn, Thivya Kandappu, Archan Misra,
• Abstract summary: EyeTrAES is a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement. We show that EyeTrAES boosts pupil tracking fidelity by 6+%, achieving IoU=92%, while incurring at least 3x lower latency than competing pure event-based eye tracking alternatives. For robust user authentication, we train a lightweight per-user Random Forest classifier using a novel feature vector of short-term pupillary kinematics.
• Score: 2.9795443606634917
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Eye-tracking technology has gained significant attention in recent years due to its wide range of applications in human-computer interaction, virtual and augmented reality, and wearable health. Traditional RGB camera-based eye-tracking systems often struggle with poor temporal resolution and computational constraints, limiting their effectiveness in capturing rapid eye movements. To address these limitations, we propose EyeTrAES, a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement that shows significant kinematic variance. One of EyeTrAES's highlights is the use of a novel adaptive windowing/slicing algorithm that ensures just the right amount of descriptive asynchronous event data accumulation within an event frame, across a wide range of eye movement patterns. EyeTrAES then applies lightweight image processing functions over accumulated event frames from just a single eye to perform pupil segmentation and tracking. We show that these methods boost pupil tracking fidelity by 6+%, achieving IoU~=92%, while incurring at least 3x lower latency than competing pure event-based eye tracking alternatives [38]. We additionally demonstrate that the microscopic pupillary motion captured by EyeTrAES exhibits distinctive variations across individuals and can thus serve as a biometric fingerprint. For robust user authentication, we train a lightweight per-user Random Forest classifier using a novel feature vector of short-term pupillary kinematics, comprising a sliding window of pupil (location, velocity, acceleration) triples. Experimental studies with two different datasets demonstrate that the EyeTrAES-based authentication technique can simultaneously achieve high authentication accuracy (~=0.82) and low processing latency (~=12ms), and significantly outperform multiple state-of-the-art competitive baselines.

Related papers

• FACET: Fast and Accurate Event-Based Eye Tracking Using Ellipse Modeling for Extended Reality [14.120171971211777]
  Event cameras offer a promising alternative due to their high temporal resolution and low power consumption. We present FACET (Fast and Accurate Event-based Eye Tracking), an end-to-end neural network that directly outputs pupil ellipse parameters from event data. On the enhanced EV-Eye test set, FACET achieves an average pupil center error of 0.20 pixels and an inference time of 0.53 ms.
  arXiv  Detail & Related papers  (2024-09-23T22:31:38Z)
• A Framework for Pupil Tracking with Event Cameras [1.708806485130162]
  Saccades are extremely rapid movements of both eyes that occur simultaneously. The peak angular speed of the eye during a saccade can reach as high as 700deg/s in humans. We present events as frames that can be readily utilized by standard deep learning algorithms.
  arXiv  Detail & Related papers  (2024-07-23T17:32:02Z)
• Apprenticeship-Inspired Elegance: Synergistic Knowledge Distillation Empowers Spiking Neural Networks for Efficient Single-Eye Emotion Recognition [53.359383163184425]
  We introduce a novel multimodality synergistic knowledge distillation scheme tailored for efficient single-eye motion recognition tasks. This method allows a lightweight, unimodal student spiking neural network (SNN) to extract rich knowledge from an event-frame multimodal teacher network.
  arXiv  Detail & Related papers  (2024-06-20T07:24:47Z)
• MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye tracking [50.26836546224782]
  Event-based eye tracking has shown great promise with the high temporal resolution and low redundancy. The diversity and abruptness of eye movement patterns, including blinking, fixating, saccades, and smooth pursuit, pose significant challenges for eye localization. This paper proposes a bidirectional long-term sequence modeling and time-varying state selection mechanism to fully utilize contextual temporal information.
  arXiv  Detail & Related papers  (2024-04-18T11:09:25Z)
• Multimodal Adaptive Fusion of Face and Gait Features using Keyless attention based Deep Neural Networks for Human Identification [67.64124512185087]
  Soft biometrics such as gait are widely used with face in surveillance tasks like person recognition and re-identification. We propose a novel adaptive multi-biometric fusion strategy for the dynamic incorporation of gait and face biometric cues by leveraging keyless attention deep neural networks.
  arXiv  Detail & Related papers  (2023-03-24T05:28:35Z)
• Recurrent Vision Transformers for Object Detection with Event Cameras [62.27246562304705]
  We present Recurrent Vision Transformers (RVTs), a novel backbone for object detection with event cameras. RVTs can be trained from scratch to reach state-of-the-art performance on event-based object detection. Our study brings new insights into effective design choices that can be fruitful for research beyond event-based vision.
  arXiv  Detail & Related papers  (2022-12-11T20:28:59Z)
• A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications [56.458448869572294]
  We introduce DETRtime, a novel framework for time-series segmentation of EEG data. Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront. Our model generalizes well in the task of EEG sleep stage segmentation.
  arXiv  Detail & Related papers  (2022-06-17T10:17:24Z)
• MTCD: Cataract Detection via Near Infrared Eye Images [69.62768493464053]
  cataract is a common eye disease and one of the leading causes of blindness and vision impairment. We present a novel algorithm for cataract detection using near-infrared eye images. Deep learning-based eye segmentation and multitask network classification networks are presented.
  arXiv  Detail & Related papers  (2021-10-06T08:10:28Z)
• Eye Know You: Metric Learning for End-to-end Biometric Authentication Using Eye Movements from a Longitudinal Dataset [4.511561231517167]
  This paper presents a convolutional neural network for authenticating users using their eye movements. The network is trained with an established metric learning loss function, multi-similarity loss. We find that eye movements are quite resilient against template aging after 3 years.
  arXiv  Detail & Related papers  (2021-04-21T12:21:28Z)
• $pi_t$- Enhancing the Precision of Eye Tracking using Iris Feature Motion Vectors [2.5889737226898437]
  A new high-precision eye-tracking method has been demonstrated recently by tracking the motion of iris features. It suffers from temporal drift, an inability to track across blinks, and loss of texture matches in the presence of motion blur. We present a new methodology $pi_t$ to address these issues by optimally combining the information from both iris textures and pupil edges.
  arXiv  Detail & Related papers  (2020-09-20T04:57:12Z)
• Differential Privacy for Eye Tracking with Temporal Correlations [30.44437258959343]
  New generation head-mounted displays, such as VR and AR glasses, are coming into the market with already integrated eye tracking. Since eye movement properties contain biometric information, privacy concerns have to be handled properly. We propose a novel transform-coding based differential privacy mechanism to further adapt it to the statistics of eye movement feature data.
  arXiv  Detail & Related papers  (2020-02-20T19:01:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.