Autonomous VR-Based Risk Detection for Situational Awareness in Dangerous Settings

• URL: http://arxiv.org/abs/2607.16582v1
• Date: Sat, 18 Jul 2026 01:48:25 GMT
• Title: Autonomous VR-Based Risk Detection for Situational Awareness in Dangerous Settings
• Abstract summary: Vision Language Models (VLMs) have shown strong potential for scene understanding in safety-critical settings.<n>We present a VR-based Human Robot Interaction framework for studying how VLM-assisted robots can support situational awareness in simulated hazardous environments.
• Score: 6.393768354249935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In high-risk environments such as disaster response, situational awareness depends not only on detecting hazards but also on communicating them clearly to human operators. Vision Language Models (VLMs) have shown strong potential for scene understanding in safety-critical settings, yet their value as part of human-facing robotic systems remains underexplored. We present a VR-based Human Robot Interaction framework for studying how VLM-assisted robots can support situational awareness in simulated hazardous environments. In our system, a robot explores a virtual scene and queries a VLM to identify potential hazards and annotate user-facing points of interest. These annotations are presented to a human operator through an immersive VR interface. This framework enables controlled evaluation of both robotic hazard identification and the communication of safety-critical information to users. Results from our study indicate that the annotated VR interface was preferred over the unannotated baseline and that participants reported high clarity, usefulness, and comfort when interacting with the system. These findings suggest that combining VLM-based robotic perception with immersive visualization is a promising approach for supporting situational awareness in hazardous settings.

Related papers

• Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses [168.50301366360344]
  Embodied Artificial Intelligence (Embodied AI) integrates perception, cognition, planning, and interaction into agents that operate in safety-critical environments.<n>Unlike digital AI systems, embodied agents must act under uncertain sensing, incomplete knowledge, and dynamic human-robot interactions.<n>This survey provides a comprehensive review of safety research in embodied AI, examining attacks and defenses across the full embodied pipeline.
  arXiv  Detail & Related papers  (2026-03-28T13:21:44Z)
• OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows [77.95511352806261]
  Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms.<n>We propose OS-Sentinel, a novel hybrid safety detection framework that combines a Formal Verifier for detecting explicit system-level violations with a Contextual Judge for assessing contextual risks and agent actions.
  arXiv  Detail & Related papers  (2025-10-28T13:22:39Z)
• ANNIE: Be Careful of Your Robots [48.89876809734855]
  We present the first systematic study of adversarial safety attacks on embodied AI systems.<n>We show attack success rates exceeding 50% across all safety categories.<n>Results expose a previously underexplored but highly consequential attack surface in embodied AI systems.
  arXiv  Detail & Related papers  (2025-09-03T15:00:28Z)
• Context-Aware Risk Estimation in Home Environments: A Probabilistic Framework for Service Robots [2.5695499302569327]
  We present a novel framework for estimating accident-prone regions in everyday indoor scenes, aimed at improving real-time risk awareness in service robots.<n>Our approach models object-level risk and context through a semantic graph-based propagation algorithm.<n>Our method is validated on a dataset with human-annotated risk regions, achieving a binary risk detection accuracy of 75%.
  arXiv  Detail & Related papers  (2025-08-27T11:14:05Z)
• Don't Let Your Robot be Harmful: Responsible Robotic Manipulation via Safety-as-Policy [53.048430683355804]
  Unthinking execution of human instructions in robotic manipulation can lead to severe safety risks.<n>We present Safety-as-policy, which includes (i) a world model to automatically generate scenarios containing safety risks and conduct virtual interactions, and (ii) a mental model to infer consequences with reflections.<n>We show that Safety-as-policy can avoid risks and efficiently complete tasks in both synthetic dataset and real-world experiments.
  arXiv  Detail & Related papers  (2024-11-27T12:27:50Z)
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics [68.36528819227641]
  This paper systematically evaluates the robustness of Vision-Language-Action (VLA) models.<n>We introduce two untargeted attack objectives that leverage spatial foundations to destabilize robotic actions, and a targeted attack objective that manipulates the robotic trajectory.<n>We design an adversarial patch generation approach that places a small, colorful patch within the camera's view, effectively executing the attack in both digital and physical environments.
  arXiv  Detail & Related papers  (2024-11-18T01:52:20Z)
• Virtual Reality and Augmented Reality Security: A Reconnaissance and Vulnerability Assessment Approach [0.0]
  Various industries have widely adopted Virtual Reality (VR) and Augmented Reality (AR) technologies to enhance productivity and user experiences. This systematic literature review focuses on identifying devices used in AR and VR technologies and specifies the associated vulnerabilities.
  arXiv  Detail & Related papers  (2024-07-22T18:51:59Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience [11.130411904676095]
  Eye tracking data, if exposed, can be used for re-identification attacks. We develop a methodology to evaluate real-time privacy mechanisms for interactive VR applications.
  arXiv  Detail & Related papers  (2024-02-12T14:53:12Z)
• Floor extraction and door detection for visually impaired guidance [78.94595951597344]
  Finding obstacle-free paths in unknown environments is a big navigation issue for visually impaired people and autonomous robots. New devices based on computer vision systems can help impaired people to overcome the difficulties of navigating in unknown environments in safe conditions. In this work it is proposed a combination of sensors and algorithms that can lead to the building of a navigation system for visually impaired people.
  arXiv  Detail & Related papers  (2024-01-30T14:38:43Z)
• Hazard recognition in an immersive virtual environment: Framework for the simultaneous analysis of visual search and EEG patterns [0.0]
  This study presents virtual safety training in an Immersive Virtual Environment (IVE) to enhance worker's hazard recognition skills. A worker wearing a Virtual Reality (VR) device, that is equipped with an eye-tracker, virtually recognizes hazards on simulated construction sites. This platform can analyze the overall performance of the workers in a visual hazard recognition task and identify hazards that need additional intervention for each worker.
  arXiv  Detail & Related papers  (2020-03-14T20:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.