Flying Together: Human-Guided Immersive Shared Control for Aerial Robot Teams in Unknown Environments

• URL: http://arxiv.org/abs/2605.21680v1
• Date: Wed, 20 May 2026 19:37:46 GMT
• Title: Flying Together: Human-Guided Immersive Shared Control for Aerial Robot Teams in Unknown Environments
• Abstract summary: We present a Virtual Reality (VR)-based shared control framework for teams of drones operating in constrained and unknown environments.<n>A novel, user-guided motion-primitive-based planner computes continuous, collision-free trajectories while continuously integrating operator input.<n>The system supports mixed-reality operations with both physical and simulated drones, and implements a bilateral VR-based interface.
• Score: 12.913935671942774
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: While autonomous multi-robots can achieve safe and coordinated navigation, they often struggle to adapt to unforeseen conditions and to capture operator-driven objectives in unstructured environments. We present a Virtual Reality (VR)-based shared control framework for teams of drones operating in constrained and unknown environments, enabling real-time, user-guided exploration. At the core of our approach is a novel, user-guided motion-primitive-based planner that computes continuous, collision-free trajectories while continuously integrating operator input. This planner is coupled with an admittance controller, allowing the operator to flexibly influence team behavior and guide drones toward regions of interest that autonomous planners may overlook. The system supports mixed-reality operations with both physical and simulated drones, and implements a bilateral VR-based interface, allowing the operator to guide the robot team via migration points while receiving immediate visual feedback of the team state. Experimental results show that shared control improves obstacle avoidance, maintains inter-agent spacing, and reduces operator effort, demonstrating the feasibility and advantages of immersive, human-in-the-loop multi-robot navigation.

Related papers

• PanoVine: Whole-Body Visuomotor Control for Soft Growing Vine Robot [13.24559047358722]
  Vine robots are suitable for navigating complex and confined environments due to their compliant bodies and self-supporting growth mechanism.<n>We present a data-driven, vision-based control framework for the first autonomous vine robot system.<n>We train an end-to-end visuomotor policy from demonstrations for closed-loop autonomous control in complex environments.
  arXiv  Detail & Related papers  (2026-06-22T07:02:19Z)
• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• A study on a Real-Time VR-Based Teleoperation Framework for Manipulator in Dynamic Environment [0.03262230127283451]
  This paper presents a VR teleoperation framework that supports real-time manipulation while handling collisions with both static and moving obstacles.<n>The framework integrates GPU-accelerated inverse kinematics and trajectory optimization within a VR interface to generate feasible joint commands.
  arXiv  Detail & Related papers  (2026-05-29T08:24:28Z)
• Self-assembling Modular Aerial Robot for Versatile Aerial Tasks [10.266801150836544]
  LEGION are reconfigurable modular aerial robots capable of in-flight self-assembly for cooperative manipulation.<n>Each unit retains nimble maneuverability while joint-equipped docking interfaces at both ends enable end-to-end self-assembly into a flying manipulator.<n>We show that multiple units autonomously dock in flight; once latched, they maintain a zero-altitude interlock by controlling the contact force and torque.
  arXiv  Detail & Related papers  (2026-05-19T06:36:10Z)
• ActiveGlasses: Learning Manipulation with Active Vision from Ego-centric Human Demonstration [51.69384671495837]
  ActiveGlasses is a system for learning robot manipulation from ego-centric human demonstrations with active vision.<n>A stereo camera mounted on smart glasses serves as the sole perception device for both data collection and policy inference.<n>To enable zero-transfer, we extract object trajectories from demonstrations and use an object-centric point-cloud policy to jointly predict manipulation and head movement.
  arXiv  Detail & Related papers  (2026-04-09T17:59:08Z)
• AURA: Multimodal Shared Autonomy for Real-World Urban Navigation [49.419374037785815]
  Long-horizon navigation in complex urban environments relies heavily on continuous human operation.<n>We present Assistive Urban Robot Autonomy (AURA), a new multi-modal framework that decomposes urban navigation into high-level human instruction and low-level AI control.<n>AURA effectively follows human instructions, reduces manual operation effort, and improves navigation stability, while enabling online adaptation.
  arXiv  Detail & Related papers  (2026-04-02T05:59:31Z)
• Navigation beyond Wayfinding: Robots Collaborating with Visually Impaired Users for Environmental Interactions [23.06831135567015]
  We present a collaborative human-robot approach that combines our robotic guide dog's precise sensing and localization capabilities with the user's ability to perform physical manipulation.<n> Evaluation results show that our system enables navigation that is safer, smoother, and more efficient than both a traditional white cane and a non-adaptive guiding system.
  arXiv  Detail & Related papers  (2026-03-15T04:21:46Z)
• Pro-HOI: Perceptive Root-guided Humanoid-Object Interaction [57.252599851031505]
  We introduce Perceptive Root-guided Humanoid-Object Interaction, Pro-HOI.<n>Pro-HOI is a generalizable framework for robust humanoid loco-manipulation.<n> Empirical validation on a Unitree G1 robot demonstrates that Pro-HOI significantly outperforms baselines in generalization and robustness.
  arXiv  Detail & Related papers  (2026-03-01T14:19:59Z)
• VLN-Pilot: Large Vision-Language Model as an Autonomous Indoor Drone Operator [1.4878644292213625]
  VLN-Pilot is a framework in which a large Vision-and-Language Model assumes the role of a human pilot for indoor drone navigation.<n>Our framework integrates language-driven semantic understanding with visual perception, enabling context-aware, high-level flight behaviors.
  arXiv  Detail & Related papers  (2026-02-05T11:23:11Z)
• Admittance-Based Motion Planning with Vision-Guided Initialization for Robotic Manipulators in Self-Driving Laboratories [0.4899818550820576]
  This paper introduces a motion-planning framework centered on admittance control to enable adaptive and compliant robotic manipulation.<n>Unlike conventional schemes, the proposed approach integrates an admittance controller directly into trajectory execution.<n>This capability enables human operators to override or redirect the robot's motion in real time.
  arXiv  Detail & Related papers  (2026-01-29T13:29:56Z)
• PILOT: A Perceptive Integrated Low-level Controller for Loco-manipulation over Unstructured Scenes [18.916978556216097]
  PILOT is a single-stage reinforcement learning framework tailored for perceptive loco-manipulation.<n>It synergizes perceptive locomotion and expansive whole-body control within a single policy.<n> PILOT demonstrates superior stability, command tracking precision, and terrain traversability compared to existing baselines.
  arXiv  Detail & Related papers  (2026-01-24T12:25:20Z)
• CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation [70.5382178207975]
  hIsight Perturbation (CHIP) is a plug-and-play module that enables controllable end-effector stiffness.<n>CHIP is easy to implement and requires neither data augmentation nor additional reward tuning.<n>We show that a generalist motion-tracking controller trained with CHIP can perform a diverse set of forceful manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-16T18:56:04Z)
• Decentralized Vision-Based Autonomous Aerial Wildlife Monitoring [55.159556673975544]
  We propose a decentralized vision-based multi-quadrotor system for wildlife monitoring.<n>Our approach enables robust identification and tracking of large species in their natural habitat.
  arXiv  Detail & Related papers  (2025-08-20T20:05:05Z)
• Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing [52.50284630866713]
  Existing systems often require hand-engineered components for state estimation, planning, and control. This paper tackles the vision-based autonomous-drone-racing problem by learning deep sensorimotor policies.
  arXiv  Detail & Related papers  (2022-10-26T19:03:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.