Low-cost Geometry-based Eye Gaze Detection using Facial Landmarks
Generated through Deep Learning
- URL: http://arxiv.org/abs/2401.00406v1
- Date: Sun, 31 Dec 2023 05:45:22 GMT
- Title: Low-cost Geometry-based Eye Gaze Detection using Facial Landmarks
Generated through Deep Learning
- Authors: Esther Enhui Ye, John Enzhou Ye, Joseph Ye, Jacob Ye, Runzhou Ye
- Abstract summary: We leverage novel face landmark detection neural networks to generate accurate and stable 3D landmarks of the face and iris.
Our approach demonstrates the ability to predict gaze with an angular error of less than 1.9 degrees, rivaling state-of-the-art systems.
- Score: 0.0937465283958018
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Introduction: In the realm of human-computer interaction and behavioral
research, accurate real-time gaze estimation is critical. Traditional methods
often rely on expensive equipment or large datasets, which are impractical in
many scenarios. This paper introduces a novel, geometry-based approach to
address these challenges, utilizing consumer-grade hardware for broader
applicability. Methods: We leverage novel face landmark detection neural
networks capable of fast inference on consumer-grade chips to generate accurate
and stable 3D landmarks of the face and iris. From these, we derive a small set
of geometry-based descriptors, forming an 8-dimensional manifold representing
the eye and head movements. These descriptors are then used to formulate linear
equations for predicting eye-gaze direction. Results: Our approach demonstrates
the ability to predict gaze with an angular error of less than 1.9 degrees,
rivaling state-of-the-art systems while operating in real-time and requiring
negligible computational resources. Conclusion: The developed method marks a
significant step forward in gaze estimation technology, offering a highly
accurate, efficient, and accessible alternative to traditional systems. It
opens up new possibilities for real-time applications in diverse fields, from
gaming to psychological research.
Related papers
- Deepfake Generation and Detection: A Benchmark and Survey [134.19054491600832]
Deepfake is a technology dedicated to creating highly realistic facial images and videos under specific conditions.
This survey comprehensively reviews the latest developments in deepfake generation and detection.
We focus on researching four representative deepfake fields: face swapping, face reenactment, talking face generation, and facial attribute editing.
arXiv Detail & Related papers (2024-03-26T17:12:34Z) - Accurate Eye Tracking from Dense 3D Surface Reconstructions using Single-Shot Deflectometry [13.297188931807586]
We propose a novel method for accurate and fast evaluation of the gaze direction that exploits teachings from single-shot phase-measuring-deflectometry(PMD)
Our method acquires dense 3D surface information of both cornea and sclera within only one single camera frame (single-shot)
We show the feasibility of our approach with experimentally evaluated gaze errors on a realistic model eye below only $0.12circ$.
arXiv Detail & Related papers (2023-08-14T17:36:39Z) - Multimodal Adaptive Fusion of Face and Gait Features using Keyless
attention based Deep Neural Networks for Human Identification [67.64124512185087]
Soft biometrics such as gait are widely used with face in surveillance tasks like person recognition and re-identification.
We propose a novel adaptive multi-biometric fusion strategy for the dynamic incorporation of gait and face biometric cues by leveraging keyless attention deep neural networks.
arXiv Detail & Related papers (2023-03-24T05:28:35Z) - FuNNscope: Visual microscope for interactively exploring the loss
landscape of fully connected neural networks [77.34726150561087]
We show how to explore high-dimensional landscape characteristics of neural networks.
We generalize observations on small neural networks to more complex systems.
An interactive dashboard opens up a number of possible application networks.
arXiv Detail & Related papers (2022-04-09T16:41:53Z) - How Facial Features Convey Attention in Stationary Environments [0.0]
This paper aims to extend previous research on distraction detection by analyzing which visual features contribute most to predicting awareness and fatigue.
We utilized the open source facial analysis toolkit OpenFace in order to analyze visual data of subjects at varying levels of attentiveness.
arXiv Detail & Related papers (2021-11-29T20:11:57Z) - Probabilistic and Geometric Depth: Detecting Objects in Perspective [78.00922683083776]
3D object detection is an important capability needed in various practical applications such as driver assistance systems.
Monocular 3D detection, as an economical solution compared to conventional settings relying on binocular vision or LiDAR, has drawn increasing attention recently but still yields unsatisfactory results.
This paper first presents a systematic study on this problem and observes that the current monocular 3D detection problem can be simplified as an instance depth estimation problem.
arXiv Detail & Related papers (2021-07-29T16:30:33Z) - Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout
for Landmark-based Facial Expression Recognition with Uncertainty Estimation [93.73198973454944]
The performance of our method is evaluated on three widely used datasets.
It is comparable to that of video-based state-of-the-art methods while it has much less complexity.
arXiv Detail & Related papers (2021-06-08T13:40:30Z) - Neural Networks for Semantic Gaze Analysis in XR Settings [0.0]
We present a novel approach which minimizes time and information necessary to annotate volumes of interest.
We train convolutional neural networks (CNNs) on synthetic data sets derived from virtual models using image augmentation techniques.
We evaluate our method in real and virtual environments, showing that the method can compete with state-of-the-art approaches.
arXiv Detail & Related papers (2021-03-18T18:05:01Z) - Towards Hardware-Agnostic Gaze-Trackers [0.5512295869673146]
We present a deep neural network architecture as an appearance-based method for constrained gaze-tracking.
Our system achieved an error of 1.8073cm on GazeCapture dataset without any calibration or device specific fine-tuning.
arXiv Detail & Related papers (2020-10-11T00:53:57Z) - Neural Topological SLAM for Visual Navigation [112.73876869904]
We design topological representations for space that leverage semantics and afford approximate geometric reasoning.
We describe supervised learning-based algorithms that can build, maintain and use such representations under noisy actuation.
arXiv Detail & Related papers (2020-05-25T17:56:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.