Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction

• URL: http://arxiv.org/abs/2605.04770v1
• Date: Wed, 06 May 2026 11:17:18 GMT
• Title: Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction
• Abstract summary: We introduce Gaze4HRI, a large-scale dataset designed to evaluate state-of-the-art performance against critical HRI variables.<n>Our benchmark reveals that all evaluated methods fail in at least one condition, identifying steeply-downward gaze as a universal failure point.
• Score: 8.819673391477036
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While zero-shot appearance-based 3D gaze estimation offers significant cost-efficiency by directly mapping RGB images to gaze vectors, its reliability in Human-Robot Interaction (HRI) settings remains uncertain. Existing benchmarks frequently overlook fundamental HRI conditions, such as dynamic camera viewpoints and moving targets in video. Furthermore, current cross-dataset evaluations often suffer from a complexity gap, where methods trained on diverse datasets are tested on significantly smaller and less varied sets, failing to assess true robustness. To bridge these gaps, we introduce Gaze4HRI, a large-scale dataset (50+ subjects, 3,000+ videos, 600,000+ frames) designed to evaluate state-of-the-art performance against critical HRI variables: illumination, head-gaze conflict, as well as the motion of camera and gaze target in video. Our benchmark reveals that all evaluated methods fail in at least one condition, identifying steeply-downward gaze as a universal failure point. Notably, PureGaze trained on the ETH-X-Gaze dataset uniquely maintains resilience across all other conditions. These results challenge the recent focus in the literature on complex spatial-temporal modeling and Transformer-based architectures. Instead, our findings suggest that extensive data diversity, as exemplified by the ETH-X-Gaze dataset, serves as the primary driver of zero-shot robustness in unconstrained environments, while resilience-enhancing frameworks, such as PureGaze's self-adversarial loss for gaze feature purification, provide a substantial further improvement. Ultimately, this study establishes a rigorous benchmark that provides practical guidelines for practitioners as well as reshaping future research. The dataset and codes are available at https://gazeforhri.github.io.

Related papers

• Event-based SLAM Benchmark for High-Speed Maneuvers [68.148886127117]
  Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution.<n>Existing event-based approaches, although successful in mitigating motion blur caused by high-speed maneuvers, suffer from many limitations.<n>We introduce a benchmarking framework for event-based state estimation, called EvSLAM, characterized by sufficient variation in data collection platforms.
  arXiv  Detail & Related papers  (2026-04-27T04:36:41Z)
• Towards Realistic Open-Vocabulary Remote Sensing Segmentation: Benchmark and Baseline [52.65099689153431]
  Open-vocabulary remote sensing image segmentation (OVRSIS) remains underexplored due to fragmented datasets, limited training diversity, and the lack of evaluation benchmarks.<n>We propose textitOVRSISBenchV2, a large-scale and application-oriented benchmark for OVRSIS.<n>Our results highlight the importance of realistic benchmark design and the effectiveness of perturbation-based transfer for OVRSIS.
  arXiv  Detail & Related papers  (2026-04-17T02:49:46Z)
• Real-time Appearance-based Gaze Estimation for Open Domains [15.195617974731766]
  We propose a robust AGE framework that enhances generalization without requiring additional human-annotated data.<n>To mitigate the impact of anisotropic inter-dataset label deviation, we reformulate gaze regression as a multi-task learning problem.<n>Our MobileNet-based lightweight model generalization achieves performance competitive with the state-of-the-art (SOTA) UniGaze-H.
  arXiv  Detail & Related papers  (2026-03-27T19:33:18Z)
• Hybrid-Domain Adaptative Representation Learning for Gaze Estimation [20.422491630669885]
  We present a novel Hybrid-domain Adaptative Representation Learning framework to learn robust gaze representation.<n>We propose to disentangle gaze-relevant representation from low-quality facial images by aligning features extracted from high-quality near-eye images.<n>Experiments on EyeDiap, MPIIFaceGaze, and Gaze360 datasets demonstrate that our approach achieves state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2025-11-17T10:38:50Z)
• OmniGaze: Reward-inspired Generalizable Gaze Estimation In The Wild [104.57404324262556]
  Current 3D gaze estimation methods struggle to generalize across diverse data domains.<n>We present OmniGaze, a semi-supervised framework for 3D gaze estimation.<n>We show that OmniGaze achieves state-of-the-art performance on five datasets.
  arXiv  Detail & Related papers  (2025-10-15T15:19:52Z)
• Bridging the Synthetic-Real Gap: Supervised Domain Adaptation for Robust Spacecraft 6-DoF Pose Estimation [13.83897333268682]
  Spacecraft Pose Estimation is a fundamental capability for autonomous space operations such as rendezvous, docking, and in-orbit docking.<n>Existing domain adaptation approaches aim to mitigate this issue but often underperform when a modest number of labeled target samples are available.<n>We propose the first Supervised Domain Adaptation (SDA) framework tailored for SPE keypoint regression.
  arXiv  Detail & Related papers  (2025-09-17T08:03:05Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• DHECA-SuperGaze: Dual Head-Eye Cross-Attention and Super-Resolution for Unconstrained Gaze Estimation [0.0]
  This paper introduces DHECA-SuperGaze, a deep learning-based method that advances gaze prediction through super-resolution (SR) and a dual head-eye cross-attention (DHECA) module.<n>Performance evaluation on Gaze360 and GFIE datasets demonstrates superior within-dataset performance of the proposed method.
  arXiv  Detail & Related papers  (2025-05-13T10:45:08Z)
• Benchmark Granularity and Model Robustness for Image-Text Retrieval [44.045767657945895]
  We show how dataset granularity and query perturbations affect retrieval performance and robustness.<n>We show that richer captions consistently enhance retrieval, especially in text-to-image tasks.<n>Our results highlight variation in model robustness and a dataset-dependent relationship between caption granularity and sensitivity perturbation.
  arXiv  Detail & Related papers  (2024-07-21T18:08:44Z)
• Exploring the Physical World Adversarial Robustness of Vehicle Detection [13.588120545886229]
  Adrial attacks can compromise the robustness of real-world detection models. We propose an innovative instant-level data generation pipeline using the CARLA simulator. Our findings highlight diverse model performances under adversarial conditions.
  arXiv  Detail & Related papers  (2023-08-07T11:09:12Z)
• OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images [59.51657161097337]
  OOD-CV-v2 is a benchmark dataset that includes out-of-distribution examples of 10 object categories in terms of pose, shape, texture, context and the weather conditions. In addition to this novel dataset, we contribute extensive experiments using popular baseline methods.
  arXiv  Detail & Related papers  (2023-04-17T20:39:25Z)
• ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation [52.5465548207648]
  ETH-XGaze is a new gaze estimation dataset consisting of over one million high-resolution images of varying gaze under extreme head poses. We show that our dataset can significantly improve the robustness of gaze estimation methods across different head poses and gaze angles.
  arXiv  Detail & Related papers  (2020-07-31T04:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.