GazeTarget360: Towards Gaze Target Estimation in 360-Degree for Robot Perception

• URL: http://arxiv.org/abs/2507.00253v1
• Date: Mon, 30 Jun 2025 20:44:40 GMT
• Title: GazeTarget360: Towards Gaze Target Estimation in 360-Degree for Robot Perception
• Authors: Zhuangzhuang Dai, Vincent Gbouna Zakka, Luis J. Manso, Chen Li,
• Abstract summary: We propose a system to address the problem of 360-degree gaze target estimation from an image.<n>The system, named GazeTarget360, integrates conditional inference engines of an eye-contact detector, a pre-trained vision encoder, and a multi-scale-fusion decoder.<n>Cross validation results show that GazeTarget360 can produce accurate and reliable gaze target predictions in unseen scenarios.
• Score: 3.312411881096304
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Enabling robots to understand human gaze target is a crucial step to allow capabilities in downstream tasks, for example, attention estimation and movement anticipation in real-world human-robot interactions. Prior works have addressed the in-frame target localization problem with data-driven approaches by carefully removing out-of-frame samples. Vision-based gaze estimation methods, such as OpenFace, do not effectively absorb background information in images and cannot predict gaze target in situations where subjects look away from the camera. In this work, we propose a system to address the problem of 360-degree gaze target estimation from an image in generalized visual scenes. The system, named GazeTarget360, integrates conditional inference engines of an eye-contact detector, a pre-trained vision encoder, and a multi-scale-fusion decoder. Cross validation results show that GazeTarget360 can produce accurate and reliable gaze target predictions in unseen scenarios. This makes a first-of-its-kind system to predict gaze targets from realistic camera footage which is highly efficient and deployable. Our source code is made publicly available at: https://github.com/zdai257/DisengageNet.

Related papers

• Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders [33.26237143983192]
  We address the problem of gaze target estimation, which aims to predict where a person is looking in a scene.<n>We propose Gaze-LLE, a novel transformer framework that streamlines gaze target estimation by leveraging features from a frozen DINOv2 encoder.
  arXiv  Detail & Related papers  (2024-12-12T18:55:30Z)
• Towards Pixel-Level Prediction for Gaze Following: Benchmark and Approach [27.84672974344777]
  We propose a novel gaze target prediction solution named GazeSeg.<n>It can fully utilize the spatial visual field of the person as guiding information and lead to a progressively coarse-to-fine gaze target segmentation and recognition process.<n>Our approach achieves the Dice of 0.325 in gaze target segmentation and 71.7% top-5 recognition.
  arXiv  Detail & Related papers  (2024-11-30T01:27:48Z)
• Stanceformer: Target-Aware Transformer for Stance Detection [59.69858080492586]
  Stance Detection involves discerning the stance expressed in a text towards a specific subject or target. Prior works have relied on existing transformer models that lack the capability to prioritize targets effectively. We introduce Stanceformer, a target-aware transformer model that incorporates enhanced attention towards the targets during both training and inference.
  arXiv  Detail & Related papers  (2024-10-09T17:24:28Z)
• Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activities [25.049754180292034]
  We address the challenge of unsupervised mistake detection in egocentric video through the analysis of gaze signals.<n>Based on the observation that eye movements closely follow object manipulation activities, we assess to what extent eye-gaze signals can support mistake detection.<n>Inconsistencies between predicted and observed gaze trajectories act as an indicator to identify mistakes.
  arXiv  Detail & Related papers  (2024-06-12T16:29:45Z)
• OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising [49.86409475232849]
  Trajectory prediction is fundamental in computer vision and autonomous driving. Existing approaches in this field often assume precise and complete observational data. We present a novel method for out-of-sight trajectory prediction that leverages a vision-positioning technique.
  arXiv  Detail & Related papers  (2024-04-02T18:30:29Z)
• Object-aware Gaze Target Detection [14.587595325977583]
  This paper proposes a Transformer-based architecture that automatically detects objects in the scene to build associations between every head and the gazed-head/object. Our method achieves state-of-the-art results on all metrics for gaze target detection and 11-13% improvement in average precision for the classification and the localization of the gazed-objects.
  arXiv  Detail & Related papers  (2023-07-18T22:04:41Z)
• Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention [44.10971508325032]
  We develop a novel model for zero-shot learning where gaze is predicted for never-before-searched objects. Gazeformer encodes the target using a natural language model, thus leveraging semantic similarities in scanpath prediction. It also outperforms existing target-detection models on standard gaze prediction for both target-present and target-absent search tasks.
  arXiv  Detail & Related papers  (2023-03-27T15:02:48Z)
• 3DGazeNet: Generalizing Gaze Estimation with Weak-Supervision from Synthetic Views [67.00931529296788]
  We propose to train general gaze estimation models which can be directly employed in novel environments without adaptation. We create a large-scale dataset of diverse faces with gaze pseudo-annotations, which we extract based on the 3D geometry of the scene. We test our method in the task of gaze generalization, in which we demonstrate improvement of up to 30% compared to state-of-the-art when no ground truth data are available.
  arXiv  Detail & Related papers  (2022-12-06T14:15:17Z)
• Active Gaze Control for Foveal Scene Exploration [124.11737060344052]
  We propose a methodology to emulate how humans and robots with foveal cameras would explore a scene. The proposed method achieves an increase in detection F1-score of 2-3 percentage points for the same number of gaze shifts.
  arXiv  Detail & Related papers  (2022-08-24T14:59:28Z)
• GIMO: Gaze-Informed Human Motion Prediction in Context [75.52839760700833]
  We propose a large-scale human motion dataset that delivers high-quality body pose sequences, scene scans, and ego-centric views with eye gaze. Our data collection is not tied to specific scenes, which further boosts the motion dynamics observed from our subjects. To realize the full potential of gaze, we propose a novel network architecture that enables bidirectional communication between the gaze and motion branches.
  arXiv  Detail & Related papers  (2022-04-20T13:17:39Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Towards End-to-end Video-based Eye-Tracking [50.0630362419371]
  Estimating eye-gaze from images alone is a challenging task due to un-observable person-specific factors. We propose a novel dataset and accompanying method which aims to explicitly learn these semantic and temporal relationships. We demonstrate that the fusion of information from visual stimuli as well as eye images can lead towards achieving performance similar to literature-reported figures.
  arXiv  Detail & Related papers  (2020-07-26T12:39:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.