Extended Reality as a Mediation Layer for Situated Human Control in Human-Robot Teaming

• URL: http://arxiv.org/abs/2607.25047v1
• Date: Mon, 27 Jul 2026 20:14:03 GMT
• Title: Extended Reality as a Mediation Layer for Situated Human Control in Human-Robot Teaming
• Abstract summary: Extended Reality (XR) is increasingly used in human-robot interaction to communicate robot intent.<n>We argue XR should also be understood as a mediation layer for situated human control in human-robot teaming.
• Score: 26.040058999347593
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extended Reality (XR) is increasingly used in human-robot interaction to communicate robot intent, planned motion, reachability, and state. We argue that XR should also be understood as a mediation layer for situated human control in human-robot teaming. Situated human control denotes the human collaborator's ability to understand, shape, authorize, and interrupt robot action within the concrete physical, social, and temporal context in which that action unfolds. We ground this perspective in scenarios from robot-assisted bedside nursing, multi-arm supervisory control, and collaborative assembly under divided attention. Across these scenarios, robot autonomy must remain inspectable and adjustable as people move, goals change, sensing is incomplete, control roles shift, and plans become invalid. We identify four mediation functions connecting human intent and robot autonomy, robot plans and human judgment, levels of shared control, and team roles, handover, and recovery. Building on these functions, we derive six design dimensions: joint action possibilities, socio-physical constraints, uncertainty and plan validity, multimodal control and correction, roles, handover, and accountability, and anticipatory recovery. The paper outlines a research agenda for XR systems that make robot autonomy more actionable and accountable in dynamic shared environments.

Related papers

• CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy [67.76658561040625]
  CHORUS is a framework that adapts a single vision-language-action (VLA) backbone to control diverse, multi-robot teams.<n>In experiments, CHORUS achieves a 64% point improvement over decentralized, from-scratch models.<n>Results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration.
  arXiv  Detail & Related papers  (2026-06-10T17:26:08Z)
• Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaboration [28.96294944351617]
  This review identifies and connects key components enabling intuitive information exchange and skill transfer between humans and robots.<n>We examine the full interaction pipeline: from the human-to-robot communication bridge translating multimodal inputs into robot-understandable representations.
  arXiv  Detail & Related papers  (2025-11-11T19:49:30Z)
• Fault-Tolerant Multi-Robot Coordination with Limited Sensing within Confined Environments [0.6144680854063939]
  We propose a novel fault-tolerance technique leveraging physical contact interactions in multi-robot systems.<n>We introduce the "Active Contact Response" (ACR) method, where each robot modulates its behavior based on the likelihood of encountering an inoperative (faulty) robot.
  arXiv  Detail & Related papers  (2025-05-21T02:43:36Z)
• HACTS: a Human-As-Copilot Teleoperation System for Robot Learning [47.9126187195398]
  We introduce HACTS (Human-As-Copilot Teleoperation System), a novel system that establishes bilateral, real-time joint synchronization between a robot arm and teleoperation hardware.<n>This simple yet effective feedback mechanism, akin to a steering wheel in autonomous vehicles, enables the human copilot to intervene seamlessly while collecting action-correction data for future learning.
  arXiv  Detail & Related papers  (2025-03-31T13:28:13Z)
• An Epistemic Human-Aware Task Planner which Anticipates Human Beliefs and Decisions [8.309981857034902]
  The aim is to build a robot policy that accounts for uncontrollable human behaviors. We propose a novel planning framework and build a solver based on AND-OR search. Preliminary experiments in two domains, one novel and one adapted, demonstrate the effectiveness of the framework.
  arXiv  Detail & Related papers  (2024-09-27T08:27:36Z)
• HARMONIC: Cognitive and Control Collaboration in Human-Robotic Teams [2.6627293764668902]
  This paper describes a cognitive-robotic architecture that integrates the OntoAgent cognitive framework with general-purpose robot control systems applied to human-robot teaming (HRT)<n>HarMONIC incorporates metacognition, meaningful natural language communication, and explainability capabilities required for developing mutual trust in HRT.
  arXiv  Detail & Related papers  (2024-09-26T16:48:21Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition [48.65867987106428]
  We introduce a novel system for joint learning between human operators and robots. It enables human operators to share control of a robot end-effector with a learned assistive agent. It reduces the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks.
  arXiv  Detail & Related papers  (2024-06-29T03:37:29Z)
• Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots [119.55240471433302]
  Habitat 3.0 is a simulation platform for studying collaborative human-robot tasks in home environments. It addresses challenges in modeling complex deformable bodies and diversity in appearance and motion. Human-in-the-loop infrastructure enables real human interaction with simulated robots via mouse/keyboard or a VR interface.
  arXiv  Detail & Related papers  (2023-10-19T17:29:17Z)
• Generalizable Human-Robot Collaborative Assembly Using Imitation Learning and Force Control [17.270360447188196]
  We present a system for human-robot collaborative assembly using learning from demonstration and pose estimation. The proposed system is demonstrated using a physical 6 DoF manipulator in a collaborative human-robot assembly scenario.
  arXiv  Detail & Related papers  (2022-12-02T20:35:55Z)
• Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together [68.44697646919515]
  This paper presents several human-robot systems that utilize spatial computing to enable novel robot use cases. The combination of spatial computing and egocentric sensing on mixed reality devices enables them to capture and understand human actions and translate these to actions with spatial meaning.
  arXiv  Detail & Related papers  (2022-02-03T10:04:26Z)
• Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration [83.4081612443128]
  We show that a short calibration using REMP can effectively bridge the gap between what a non-expert user thinks a robot can reach and the ground-truth. We show that this calibration procedure not only results in better user perception, but also promotes more efficient human-robot collaborations.
  arXiv  Detail & Related papers  (2021-03-06T09:14:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.