Low Latency Gaze Tracking via Latent Optical Sensing

• URL: http://arxiv.org/abs/2605.17990v1
• Date: Mon, 18 May 2026 07:46:25 GMT
• Title: Low Latency Gaze Tracking via Latent Optical Sensing
• Abstract summary: We present a real-time gaze tracking system that directly acquires task-relevant latent features using a fully passive optical encoder.<n>We demonstrate the effectiveness of our approach on both simulated and real-world data, achieving competitive gaze estimation accuracy.<n>This work highlights the potential of task-driven optical sensing for ultra-low-latency, computationally efficient human-computer interaction systems.
• Score: 12.829480947195412
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a real-time gaze tracking system that directly acquires task-relevant latent features using a fully passive optical encoder. Instead of forming and processing full-resolution images, our approach leverages a microlens array with a co-designed binary chromium mask to perform spatially multiplexed optical encoding, producing a compact set of measurements sufficient for gaze estimation. By integrating sensing and feature extraction in the optical domain, the proposed system eliminates the need for high-bandwidth image readout and substantially reduces computational overhead. The encoded measurements are captured by a 4 x 4 phototransistor array and mapped to gaze direction using a lightweight neural network. Our proof-of-concept prototype enables an end-to-end sensing-to-inference latency of 3.4 ms, outperforming published research systems. We demonstrate the effectiveness of our approach on both simulated and real-world data, achieving competitive gaze estimation accuracy while significantly improving latency and energy efficiency compared to conventional camera-based pipelines. This work highlights the potential of task-driven optical sensing for ultra-low-latency, computationally efficient human-computer interaction systems.

Related papers

• Optical Flow Sensor: A Direction-Selective Bionic Retina Design [1.3997127824012905]
  pixel-level Optical Flow Sensor (OFS) integrated circuit.<n>Design combines Dynamic Vision Sensor (DVS) ON/OFF event comparison with time-difference measurement to enable fully parallel optical flow computation on-chip.
  arXiv  Detail & Related papers  (2026-07-30T04:59:33Z)
• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• Nonlinear optical encoding enabled by recurrent linear scattering [16.952531256252744]
  We introduce a design that passively induce optical nonlinear random mapping with a continuous-wave laser at a low power.<n>We demonstrate that our design retains vital information even when the readout dimensionality is reduced.<n>This capability allows our optical platforms to offer efficient optical information processing solutions across applications.
  arXiv  Detail & Related papers  (2023-07-17T15:15:47Z)
• Neuromorphic Optical Flow and Real-time Implementation with Event Cameras [47.11134388304464]
  We build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture that improves the state-of-the-art self-supervised optical flow accuracy. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity.
  arXiv  Detail & Related papers  (2023-04-14T14:03:35Z)
• Scattering-induced entropy boost for highly-compressed optical sensing and encryption [7.502671257653539]
  Image sensing often relies on a high-quality machine vision system with a large field of view and high resolution. We propose a novel image-free sensing framework for resource-efficient image classification. The proposed framework is shown to obtain over a 95% accuracy at sampling rates of 1% and 5% for classification on the MNIST dataset.
  arXiv  Detail & Related papers  (2022-12-16T09:00:42Z)
• Sensor-Guided Optical Flow [53.295332513139925]
  This paper proposes a framework to guide an optical flow network with external cues to achieve superior accuracy on known or unseen domains. We show how these can be obtained by combining depth measurements from active sensors with geometry and hand-crafted optical flow algorithms.
  arXiv  Detail & Related papers  (2021-09-30T17:59:57Z)
• Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo [61.6260594326246]
  We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
  arXiv  Detail & Related papers  (2021-03-22T18:06:58Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)
• Fast and Accurate Light Field Saliency Detection through Deep Encoding [0.8356765961526955]
  Light field saliency detection still lacks speed and can improve in accuracy. Existing approaches consume unnecessarily large amounts of computational resources for training, and have longer execution times for testing. We solve this by aggressively reducing the large light field images to a much smaller three-channel feature map.
  arXiv  Detail & Related papers  (2020-10-25T09:15:08Z)
• Rapid characterisation of linear-optical networks via PhaseLift [51.03305009278831]
  Integrated photonics offers great phase-stability and can rely on the large scale manufacturability provided by the semiconductor industry. New devices, based on such optical circuits, hold the promise of faster and energy-efficient computations in machine learning applications. We present a novel technique to reconstruct the transfer matrix of linear optical networks.
  arXiv  Detail & Related papers  (2020-10-01T16:04:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.