Dual-Path Enhancements in Event-Based Eye Tracking: Augmented Robustness and Adaptive Temporal Modeling

• URL: http://arxiv.org/abs/2504.09960v1
• Date: Mon, 14 Apr 2025 07:57:22 GMT
• Title: Dual-Path Enhancements in Event-Based Eye Tracking: Augmented Robustness and Adaptive Temporal Modeling
• Abstract summary: Event-based eye tracking has become a pivotal technology for augmented reality and human-computer interaction.<n>Existing methods struggle with real-world challenges such as abrupt eye movements and environmental noise.<n>We introduce two key advancements. First, a robust data augmentation pipeline incorporating temporal shift, spatial flip, and event deletion improves model resilience.<n>Second, we propose KnightPupil, a hybrid architecture combining an EfficientNet-B3 backbone for spatial feature extraction, a bidirectional GRU for contextual temporal modeling, and a Linear Time-Varying State-Space Module.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Event-based eye tracking has become a pivotal technology for augmented reality and human-computer interaction. Yet, existing methods struggle with real-world challenges such as abrupt eye movements and environmental noise. Building on the efficiency of the Lightweight Spatiotemporal Network-a causal architecture optimized for edge devices-we introduce two key advancements. First, a robust data augmentation pipeline incorporating temporal shift, spatial flip, and event deletion improves model resilience, reducing Euclidean distance error by 12% (1.61 vs. 1.70 baseline) on challenging samples. Second, we propose KnightPupil, a hybrid architecture combining an EfficientNet-B3 backbone for spatial feature extraction, a bidirectional GRU for contextual temporal modeling, and a Linear Time-Varying State-Space Module to adapt to sparse inputs and noise dynamically. Evaluated on the 3ET+ benchmark, our framework achieved 1.61 Euclidean distance on the private test set of the Event-based Eye Tracking Challenge at CVPR 2025, demonstrating its effectiveness for practical deployment in AR/VR systems while providing a foundation for future innovations in neuromorphic vision.

Related papers

• Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation [68.02899606570223]
  We propose TAMP-Nav, a unified framework for efficient embodied navigation.<n>First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual coordinates.<n>Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize)
  arXiv  Detail & Related papers  (2026-08-18T08:37:08Z)
• DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks [54.32016216994156]
  We introduce the Causal Latent World Model (CLWM), which employs DINOv3 features as generative targets to disentangle interaction semantics from visual noise.<n>To overcome memory scaling, CLWM features a Dual-State Test-Time Training (TTT) Memory that guarantees a strict $mathcalO(1)$ footprint for long-horizon tasks.<n>We present EmbodiChain, an online framework that establishes the Efficiency Law by injecting an infinite flow of physics-grounded trajectories during training.
  arXiv  Detail & Related papers  (2026-04-13T03:19:36Z)
• Lightweight Spatiotemporal Highway Lane Detection via 3D-ResNet and PINet with ROI-Aware Attention [0.5156484100374058]
  We propose two models that integrate a 3D-ResNet encoder with a Point Instance Network (PINet) decoder.<n>The architecture has been validated through offline training and real-time inference in the Autonomous Systems Laboratory at City, St George's University of London.
  arXiv  Detail & Related papers  (2026-04-02T15:47:16Z)
• SMV-EAR: Bring Spatiotemporal Multi-View Representation Learning into Efficient Event-Based Action Recognition [4.322175390073132]
  Event action recognition (EAR) offers privacy-protecting and efficiency advantages where temporal motion dynamics is of great importance.<n>This paper reexamines the key SMVRL design stages for EAR and proposes a principled multi-view representation through translation-invariant dense conversion of sparse events.<n>We show that Top-1 accuracy gains over existing SMVRL EOR method with surprising 30.1% reduced parameters and 30.2% lower computations, establishing our framework as a novel and powerful EAR paradigm.
  arXiv  Detail & Related papers  (2026-01-24T09:24:42Z)
• Neuromorphic Eye Tracking for Low-Latency Pupil Detection [37.091037454305024]
  Eye tracking for wearable systems demands low latency and milliwatt-level power.<n>Neuromorphic sensors and spiking neural networks (SNNs) offer a promising alternative.<n>This paper presents a neuromorphic version of top-performing event-based eye-tracking models.
  arXiv  Detail & Related papers  (2025-12-10T11:30:21Z)
• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• Efficient Spatial-Temporal Modeling for Real-Time Video Analysis: A Unified Framework for Action Recognition and Object Tracking [0.0]
  Real-time video analysis remains a challenging problem in computer vision.<n>We present a unified framework that leverages advanced spatial-temporal modeling techniques for simultaneous action recognition and object tracking.<n>Our method achieves state-of-the-art performance on standard benchmarks while maintaining real-time inference speeds.
  arXiv  Detail & Related papers  (2025-07-30T06:49:11Z)
• S3MOT: Monocular 3D Object Tracking with Selective State Space Model [3.5047603107971397]
  Multi-object tracking in 3D space is essential for advancing robotics and computer applications. It remains a significant challenge in monocular setups due to the difficulty of mining 3D associations from 2D video streams. We present three innovative techniques to enhance the fusion of heterogeneous cues for monocular 3D MOT.
  arXiv  Detail & Related papers  (2025-04-25T04:45:35Z)
• Spatiotemporal Attention Learning Framework for Event-Driven Object Recognition [1.0445957451908694]
  Event-based vision sensors capture local pixel-level intensity changes as a sparse event stream containing position, polarity, and information.<n>This paper presents a novel learning framework for event-based object recognition, utilizing a VARGG network enhanced with Contemporalal Block Attention Module (CBAM)<n>Our approach achieves comparable performance to state-of-the-art ResNet-based methods while reducing parameter count by 2.3% compared to the original VGG model.
  arXiv  Detail & Related papers  (2025-04-01T02:37:54Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• OccLinker: Deflickering Occupancy Networks through Lightweight Spatio-Temporal Correlation [15.726401007342087]
  Vision-based occupancy networks (VONs) provide an end-to-end solution for reconstructing 3D environments in autonomous driving.<n>Recent approaches have incorporated historical data to mitigate the issue, but they often incur high computational costs and introduce noisy information that interferes with object detection.<n>We propose OccLinker, a novel plugin framework designed to seamlessly integrate with existing VONs for boosting performance.
  arXiv  Detail & Related papers  (2025-02-21T13:07:45Z)
• You Only Crash Once v2: Perceptually Consistent Strong Features for One-Stage Domain Adaptive Detection of Space Terrain [4.339510167603377]
  In-situ detection of planetary, lunar, and small-body surface terrain is crucial for autonomous spacecraft applications.<n>Unsupervised Domain Adaptation (UDA) offers a promising solution by facilitating model training with disparate data sources.<n>We propose novel additions to the VSA scheme that enhance terrain detection capabilities under UDA.
  arXiv  Detail & Related papers  (2025-01-23T14:58:49Z)
• DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes [71.61083731844282]
  We present DeSiRe-GS, a self-supervised gaussian splatting representation.<n>It enables effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios.
  arXiv  Detail & Related papers  (2024-11-18T05:49:16Z)
• ALOcc: Adaptive Lifting-based 3D Semantic Occupancy and Cost Volume-based Flow Prediction [89.89610257714006]
  Existing methods prioritize higher accuracy to cater to the demands of these tasks. We introduce a series of targeted improvements for 3D semantic occupancy prediction and flow estimation. Our purelytemporalal architecture framework, named ALOcc, achieves an optimal tradeoff between speed and accuracy.
  arXiv  Detail & Related papers  (2024-11-12T11:32:56Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• A Lightweight Spatiotemporal Network for Online Eye Tracking with Event Camera [0.8576354642891824]
  Event-based data are commonly encountered in edge computing environments where efficiency and low latency are critical. To interface with such data and leverage their rich temporal temporal, we propose a causal convolutional network. We apply our model on the AIS 2024 event-based eye tracking challenge, reaching a score of 0.9916 p10 accuracy on the Kaggle private testset.
  arXiv  Detail & Related papers  (2024-04-13T00:13:20Z)
• Recurrent Vision Transformers for Object Detection with Event Cameras [62.27246562304705]
  We present Recurrent Vision Transformers (RVTs), a novel backbone for object detection with event cameras. RVTs can be trained from scratch to reach state-of-the-art performance on event-based object detection. Our study brings new insights into effective design choices that can be fruitful for research beyond event-based vision.
  arXiv  Detail & Related papers  (2022-12-11T20:28:59Z)
• Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networks [110.80088437391379]
  A graph-based framework called SMART is proposed to model and keep track of the statistics of vehicle-to-temporal (V2I) communication latency across a large geographical area. We develop a graph reconstruction-based approach using a graph convolutional network integrated with a deep Q-networks algorithm. Our results show that the proposed method can significantly improve both the accuracy and efficiency for modeling and the latency performance of large vehicular networks.
  arXiv  Detail & Related papers  (2021-03-13T06:56:29Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.