Will You Be Aware? Eye Tracking-Based Modeling of Situational Awareness in Augmented Reality

• URL: http://arxiv.org/abs/2508.05025v2
• Date: Tue, 02 Sep 2025 02:29:00 GMT
• Title: Will You Be Aware? Eye Tracking-Based Modeling of Situational Awareness in Augmented Reality
• Authors: Zhehan Qu, Tianyi Hu, Christian Fronk, Maria Gorlatova,
• Abstract summary: Augmented Reality (AR) systems pose risks of inducing cognitive tunneling-a hyperfocus on virtual content that compromises situational awareness (SA) in safety-critical scenarios.<n>This paper investigates SA in AR-guided resuscitation (CPR), where responders must balance effective compressions with vigilance to unpredictable cardiopulmonary hazards.<n>We developed an AR app that overlays real-time CPR feedback (compression depth and rate) and conducted a user study with simulated unexpected incidents.<n>Eye tracking analysis revealed that higher SA levels were associated with greater saccadic amplitude and velocity, and with reduced proportion and frequency of fixations on virtual content.
• Score: 7.811210736680597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Augmented Reality (AR) systems, while enhancing task performance through real-time guidance, pose risks of inducing cognitive tunneling-a hyperfocus on virtual content that compromises situational awareness (SA) in safety-critical scenarios. This paper investigates SA in AR-guided cardiopulmonary resuscitation (CPR), where responders must balance effective compressions with vigilance to unpredictable hazards (e.g., patient vomiting). We developed an AR app on a Magic Leap 2 that overlays real-time CPR feedback (compression depth and rate) and conducted a user study with simulated unexpected incidents (e.g., bleeding) to evaluate SA, in which SA metrics were collected via observation and questionnaires administered during freeze-probe events. Eye tracking analysis revealed that higher SA levels were associated with greater saccadic amplitude and velocity, and with reduced proportion and frequency of fixations on virtual content. To predict SA, we propose FixGraphPool, a graph neural network that structures gaze events (fixations, saccades) into spatiotemporal graphs, effectively capturing dynamic attentional patterns. Our model achieved 83.0% accuracy (F1=81.0%), outperforming feature-based machine learning and state-of-the-art time-series models by leveraging domain knowledge and spatial-temporal information encoded in ET data. These findings demonstrate the potential of eye tracking for SA modeling in AR and highlight its utility in designing AR systems that ensure user safety and situational awareness.

Related papers

• AURA: Development and Validation of an Augmented Unplanned Removal Alert System using Synthetic ICU Videos [1.8993428741072542]
  Unplanned extubation (UE) remains a critical patient safety concern in intensive care units (ICUs)<n>We propose Augmented Unplanned Removal Alert (AURA), a vision-based risk detection system developed and validated entirely on a fully synthetic video dataset.
  arXiv  Detail & Related papers  (2025-11-15T14:52:37Z)
• DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space [60.729377189859]
  We propose our DAWP framework to enable AIWPs to operate in a complete observation space.<n>AIDA module applies a mask multi-modality autoencoder for assimilating irregular satellite observation tokens.<n>We show that AIDA significantly improves the roll out and efficiency of AIWP and holds promising potential to be applied in global precipitationresolution forecasting.
  arXiv  Detail & Related papers  (2025-10-13T03:13:35Z)
• Unleashing Perception-Time Scaling to Multimodal Reasoning Models [60.578179197783754]
  Recent advances in inference-time scaling have substantially enhanced the reasoning capabilities of Large Vision-Language Models (LVLMs)<n>Inspired by this success, similar strategies have been applied to multimodal reasoning, yet their impact on visual perception remains unclear.<n>We propose Perception-Time Scaling (PTS), a novel paradigm that encourages token-rich perception and decomposes complex perception problems into intermediate tractable sub-problems.
  arXiv  Detail & Related papers  (2025-10-10T03:17:52Z)
• Real-Time Assessment of Bystander Situation Awareness in Drone-Assisted First Aid [6.986130179800924]
  Rapid naloxone delivery via drones offers a promising solution for responding to opioid overdose emergencies (OOEs)<n>We address the critical role of bystander situational awareness (SA) in human-autonomy teaming (HAT)<n>We propose a video-based real-time SA assessment framework that utilizes graph embeddings and transformer models to assess bystander SA in real time.
  arXiv  Detail & Related papers  (2025-10-03T23:11:30Z)
• Action Recognition in Real-World Ambient Assisted Living Environment [1.0787328610467801]
  Ambient Assisted Living technologies facilitate ageing in place by offering continuous monitoring and assistance within the home.<n>Within AAL technologies, action recognition plays a crucial role in interpreting human activities.<n>This paper introduces the Robust and Efficient Temporal Convolution network (RE-TCN)
  arXiv  Detail & Related papers  (2025-03-29T20:32:22Z)
• Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks [57.17129753411926]
  Spiking neural networks (SNNs) are emerging as a promising alternative to traditional artificial neural networks (ANNs)<n>We propose SpikeSR, which achieves state-of-the-art performance across various remote sensing benchmarks such as AID, DOTA, and DIOR.
  arXiv  Detail & Related papers  (2025-03-06T09:06:06Z)
• Salient Information Preserving Adversarial Training Improves Clean and Robust Accuracy [4.737519767218666]
  We introduce Salient Information Preserving Adversarial Training (SIP-AT)<n>SIP-AT is an intuitive method for relieving the robustness-accuracy trade-off incurred by traditional adversarial training.<n>We demonstrate that SIP-AT is able to boost the clean accuracy of models while maintaining a high degree of robustness against attacks at multiple epsilon levels.
  arXiv  Detail & Related papers  (2025-01-15T19:12:59Z)
• Psych-Occlusion: Using Visual Psychophysics for Aerial Detection of Occluded Persons during Search and Rescue [41.03292974500013]
  Small Unmanned Aerial Systems (sUAS) as "eyes in the sky" during Emergency Response (ER) scenarios.<n> efficient detection of persons from aerial views plays a crucial role in achieving a successful mission outcome.<n>Performance of Computer Vision (CV) models onboard sUAS substantially degrades under real-life rigorous conditions.<n>We exemplify the use of our behavioral dataset, Psych-ER, by using its human accuracy data to adapt the loss function of a detection model.
  arXiv  Detail & Related papers  (2024-12-07T06:22:42Z)
• Towards Evaluating the Robustness of Visual State Space Models [63.14954591606638]
  Vision State Space Models (VSSMs) have demonstrated remarkable performance in visual perception tasks. However, their robustness under natural and adversarial perturbations remains a critical concern. We present a comprehensive evaluation of VSSMs' robustness under various perturbation scenarios.
  arXiv  Detail & Related papers  (2024-06-13T17:59:44Z)
• REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates [54.96885726053036]
  This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks.
  arXiv  Detail & Related papers  (2024-06-03T16:30:19Z)
• Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features [50.82725748981231]
  Engagement measurement finds application in healthcare, education, services. Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors. We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
  arXiv  Detail & Related papers  (2024-04-05T20:39:16Z)
• Two-Stage Violence Detection Using ViTPose and Classification Models at Smart Airports [9.53984191161849]
  This study introduces an innovative violence detection framework tailored to the unique requirements of smart airports. The framework harnesses the power of ViTPose for human pose estimation. The solution underwent integrated testing to ensure robust performance in real world scenarios.
  arXiv  Detail & Related papers  (2023-08-30T21:20:15Z)
• Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions [18.545193011418654]
  We look at the generalization capabilities of visual question answering (VQA) systems. We propose a new robustness measure, Robustness to Augmented Data (RAD), which measures the consistency of model predictions. We find substantial failure cases which reveal that current VQA systems are still brittle.
  arXiv  Detail & Related papers  (2021-06-08T16:09:47Z)
• Federated Learning in the Sky: Aerial-Ground Air Quality Sensing Framework with UAV Swarms [53.38353133198842]
  Air quality significantly affects human health, it is increasingly important to accurately and timely predict the Air Quality Index (AQI) This paper proposes a new federated learning-based aerial-ground air quality sensing framework for fine-grained 3D air quality monitoring and forecasting. For ground sensing systems, we propose a Graph Convolutional neural network-based Long Short-Term Memory (GC-LSTM) model to achieve accurate, real-time and future AQI inference.
  arXiv  Detail & Related papers  (2020-07-23T13:32:47Z)
• Adversarial vs behavioural-based defensive AI with joint, continual and active learning: automated evaluation of robustness to deception, poisoning and concept drift [62.997667081978825]
  Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security. In this paper, we present a solution to effectively mitigate this attack by improving the detection process and efficiently leveraging human expertise.
  arXiv  Detail & Related papers  (2020-01-13T13:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.