Validating Virtual Reality for Studying Multimodal Human-Robot Interaction in Socially Aware Robot Navigation

• URL: http://arxiv.org/abs/2607.09261v1
• Date: Fri, 10 Jul 2026 10:23:57 GMT
• Title: Validating Virtual Reality for Studying Multimodal Human-Robot Interaction in Socially Aware Robot Navigation
• Abstract summary: We present a VR prototype and evaluate its suitability for studying multimodal human-robot interaction (HRI) in socially aware navigation.<n>Specifically, we investigate whether VR preserves the multimodal interaction dynamics observed in real-world human-robot co-navigation.<n>Our results show that participants perceive the robot's socially aware navigation similarly in VR and in the real world.
• Score: 5.566807756855081
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Virtual Reality (VR) offers a flexible and controllable platform for studying human-robot interaction. Prior work has explored VR for socially aware robot navigation. However, whether VR captures the multimodal interaction dynamics observed in real-world human-robot co-navigation remains insufficiently understood. In this work, we present a VR prototype and evaluate its suitability for studying multimodal human-robot interaction (HRI) in socially aware navigation. Specifically, we investigate whether VR preserves the multimodal interaction dynamics observed in real-world human-robot co-navigation. We conducted a within-subjects study (N = 21) in which participants interacted with a PR2 mobile manipulator robot in both a motion capture equipped arena and its virtual replica in an immersive VR environment. Two common co-navigation scenarios were examined : orthogonal crossing and pass-by interactions. Participants evaluated the robot's perceived social awareness and interaction comfort, while trajectory and head-orientation data were analysed to examine behavioral responses during the interaction. Our results show that participants perceive the robot's socially aware navigation similarly in VR and in the real world. Furthermore, VR captures human interaction behaviors in ways consistent with real-world observations. These findings suggest that VR can be a reliable and flexible platform for studying richer multimodal behaviors in social navigation and HRI.

Related papers

• XR$^3$: An Extended Reality Platform for Social-Physical Human-Robot Interaction [7.5509967369489965]
  We present XR$3$, a co-located operator and a hidden dual-VR-headset platform for HRI research.<n>The robot's upper-body motion, head and gaze behavior, and facial expressions are mapped from the operator's tracked limbs and face signals.<n>We detail the system design and calibration, and demonstrate the platform in a touch-based Wizard-of-Oz study.
  arXiv  Detail & Related papers  (2026-01-18T13:11:41Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• Recognizing Actions from Robotic View for Natural Human-Robot Interaction [52.00935005918032]
  Natural Human-Robot Interaction (N-HRI) requires robots to recognize human actions at varying distances and states, regardless of whether the robot itself is in motion or stationary.<n>Existing benchmarks for N-HRI fail to address the unique complexities in N-HRI due to limited data, modalities, task categories, and diversity of subjects and environments.<n>We introduce (Action from Robotic View) a large-scale dataset for perception-centric robotic views prevalent in mobile service robots.
  arXiv  Detail & Related papers  (2025-07-30T09:48:34Z)
• Behavioural gap assessment of human-vehicle interaction in real and virtual reality-based scenarios in autonomous driving [7.588679613436823]
  We present a first and innovative approach to evaluating what we term the behavioural gap, a concept that captures the disparity in a participant's conduct when engaging in a VR experiment compared to an equivalent real-world situation. In the experiment, the pedestrian attempts to cross the road in the presence of different driving styles and an external Human-Machine Interface (eHMI) Results show that participants are more cautious and curious in VR, affecting their speed and decisions, and that VR interfaces significantly influence their actions.
  arXiv  Detail & Related papers  (2024-07-04T17:20:17Z)
• Tremor Reduction for Accessible Ray Based Interaction in VR Applications [0.0]
  Many traditional 2D interface interaction methods have been directly converted to work in a VR space with little alteration to the input mechanism. In this paper we propose the use of a low pass filter, to normalize user input noise, alleviating fine motor requirements during ray-based interaction.
  arXiv  Detail & Related papers  (2024-05-12T17:07:16Z)
• VR interaction for efficient virtual manufacturing: mini map for multi-user VR navigation platform [0.0]
  This paper focuses on interactive positioning maps for virtual factory layout planning and exploring the user interaction design of digital maps as navigation aid. Five different prototypes of interactive maps were tested, evaluated and graded by the 20 participants and 40 validated data streams collected. The most efficient interaction design of interactive maps is thus analyzed and discussed in the study.
  arXiv  Detail & Related papers  (2023-12-29T13:09:54Z)
• Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots [119.55240471433302]
  Habitat 3.0 is a simulation platform for studying collaborative human-robot tasks in home environments. It addresses challenges in modeling complex deformable bodies and diversity in appearance and motion. Human-in-the-loop infrastructure enables real human interaction with simulated robots via mouse/keyboard or a VR interface.
  arXiv  Detail & Related papers  (2023-10-19T17:29:17Z)
• SACSoN: Scalable Autonomous Control for Social Navigation [62.59274275261392]
  We develop methods for training policies for socially unobtrusive navigation. By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space. We collect a large dataset where an indoor mobile robot interacts with human bystanders.
  arXiv  Detail & Related papers  (2023-06-02T19:07:52Z)
• Learning Effect of Lay People in Gesture-Based Locomotion in Virtual Reality [81.5101473684021]
  Some of the most promising methods are gesture-based and do not require additional handheld hardware. Recent work focused mostly on user preference and performance of the different locomotion techniques. This work is investigated whether and how quickly users can adapt to a hand gesture-based locomotion system in VR.
  arXiv  Detail & Related papers  (2022-06-16T10:44:16Z)
• Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation [92.66286342108934]
  Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a'socially compliant' manner in the presence of other intelligent agents such as humans. Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human teleoperated driving demonstrations.
  arXiv  Detail & Related papers  (2022-03-28T19:09:11Z)
• Visual Navigation Among Humans with Optimal Control as a Supervisor [72.5188978268463]
  We propose an approach that combines learning-based perception with model-based optimal control to navigate among humans. Our approach is enabled by our novel data-generation tool, HumANav. We demonstrate that the learned navigation policies can anticipate and react to humans without explicitly predicting future human motion.
  arXiv  Detail & Related papers  (2020-03-20T16:13:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.