Related papers: A Framework for Pupil Tracking with Event Cameras

A Framework for Pupil Tracking with Event Cameras

URL: http://arxiv.org/abs/2407.16665v2
Date: Mon, 7 Oct 2024 09:46:07 GMT
Title: A Framework for Pupil Tracking with Event Cameras
Authors: Khadija Iddrisu, Waseem Shariff, Suzanne Little,
Abstract summary: Saccades are extremely rapid movements of both eyes that occur simultaneously. The peak angular speed of the eye during a saccade can reach as high as 700deg/s in humans. We present events as frames that can be readily utilized by standard deep learning algorithms.
Score: 1.708806485130162
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Saccades are extremely rapid movements of both eyes that occur simultaneously, typically observed when an individual shifts their focus from one object to another. These movements are among the swiftest produced by humans and possess the potential to achieve velocities greater than that of blinks. The peak angular speed of the eye during a saccade can reach as high as 700{\deg}/s in humans, especially during larger saccades that cover a visual angle of 25{\deg}. Previous research has demonstrated encouraging outcomes in comprehending neurological conditions through the study of saccades. A necessary step in saccade detection involves accurately identifying the precise location of the pupil within the eye, from which additional information such as gaze angles can be inferred. Conventional frame-based cameras often struggle with the high temporal precision necessary for tracking very fast movements, resulting in motion blur and latency issues. Event cameras, on the other hand, offer a promising alternative by recording changes in the visual scene asynchronously and providing high temporal resolution and low latency. By bridging the gap between traditional computer vision and event-based vision, we present events as frames that can be readily utilized by standard deep learning algorithms. This approach harnesses YOLOv8, a state-of-the-art object detection technology, to process these frames for pupil tracking using the publicly accessible Ev-Eye dataset. Experimental results demonstrate the framework's effectiveness, highlighting its potential applications in neuroscience, ophthalmology, and human-computer interaction.

Related papers

EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing [2.9795443606634917]
EyeTrAES is a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement. We show that EyeTrAES boosts pupil tracking fidelity by 6+%, achieving IoU=92%, while incurring at least 3x lower latency than competing pure event-based eye tracking alternatives. For robust user authentication, we train a lightweight per-user Random Forest classifier using a novel feature vector of short-term pupillary kinematics.
arXiv Detail & Related papers (2024-09-27T15:06:05Z)
Computer Vision for Primate Behavior Analysis in the Wild [61.08941894580172]
Video-based behavioral monitoring has great potential for transforming how we study animal cognition and behavior. There is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today.
arXiv Detail & Related papers (2024-01-29T18:59:56Z)
In the Blink of an Eye: Event-based Emotion Recognition [44.12621619057609]
We introduce a wearable single-eye emotion recognition device and a real-time approach to recognizing emotions from partial observations of an emotion. At the heart of our method is a bio-inspired event-based camera setup and a newly designed lightweight Spiking Eye Emotion Network (SEEN)
arXiv Detail & Related papers (2023-10-06T06:33:20Z)
Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks [55.81577205593956]
Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously. Deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential.
arXiv Detail & Related papers (2023-02-17T14:19:28Z)
A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications [56.458448869572294]
We introduce DETRtime, a novel framework for time-series segmentation of EEG data. Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront. Our model generalizes well in the task of EEG sleep stage segmentation.
arXiv Detail & Related papers (2022-06-17T10:17:24Z)
GIMO: Gaze-Informed Human Motion Prediction in Context [75.52839760700833]
We propose a large-scale human motion dataset that delivers high-quality body pose sequences, scene scans, and ego-centric views with eye gaze. Our data collection is not tied to specific scenes, which further boosts the motion dynamics observed from our subjects. To realize the full potential of gaze, we propose a novel network architecture that enables bidirectional communication between the gaze and motion branches.
arXiv Detail & Related papers (2022-04-20T13:17:39Z)
Head and eye egocentric gesture recognition for human-robot interaction using eyewear cameras [4.344337854565144]
This work addresses the problem of human gesture recognition. In particular, we focus on head and eye gestures, and adopt an egocentric (first-person) perspective using eyewear cameras. A motion-based recognition approach is proposed, which operates at two temporal granularities.
arXiv Detail & Related papers (2022-01-27T13:26:05Z)
Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations [75.60524561611008]
This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
arXiv Detail & Related papers (2021-09-28T20:25:10Z)
GazeMAE: General Representations of Eye Movements using a Micro-Macro Autoencoder [0.0]
We propose an abstract representation of eye movements that preserve the important nuances in gaze behavior while being stimuli-agnostic. We consider eye movements as raw position and velocity signals and train separate deep temporal convolutional autoencoders. The autoencoders learn micro-scale and macro-scale representations that correspond to the fast and slow features of eye movements.
arXiv Detail & Related papers (2020-09-05T02:13:42Z)
Real-Time Face and Landmark Localization for Eyeblink Detection [1.617280000789628]
A face- and landmark-detection algorithm have been carefully combined in order to provide fully automated eyelid tracking. Our experimental evaluation reveals an overall runtime application of 0.533 ms per frame. This is 1,101$times$ faster than the sequential implementation and well within the real-time requirements of eyeblink conditioning in humans.
arXiv Detail & Related papers (2020-06-01T09:46:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.