A study on a Real-Time VR-Based Teleoperation Framework for Manipulator in Dynamic Environment

• URL: http://arxiv.org/abs/2605.30989v1
• Date: Fri, 29 May 2026 08:24:28 GMT
• Title: A study on a Real-Time VR-Based Teleoperation Framework for Manipulator in Dynamic Environment
• Abstract summary: This paper presents a VR teleoperation framework that supports real-time manipulation while handling collisions with both static and moving obstacles.<n>The framework integrates GPU-accelerated inverse kinematics and trajectory optimization within a VR interface to generate feasible joint commands.
• Score: 0.03262230127283451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robot teleoperation enables safe, non-contact task execution in hazardous environments where direct human access is difficult, and its application has expanded with recent VR technologies. Many VR teleoperation studies, however, have primarily served as data-collection tools for robot imitation learning, so they often do not explicitly address dynamic obstacles, workspace changes, or collision risks during operation. For real deployment aimed at operator safety, teleoperation must react to dynamic situations with low latency and remain robust to mistakes made by inexperienced operators. This paper presents a VR teleoperation framework that supports real-time manipulation while handling collisions with both static and moving obstacles. The framework integrates GPU-accelerated inverse kinematics and trajectory optimization within a VR interface to generate feasible joint commands at each control cycle under robot constraints. Experiments with a 7-DoF manipulator demonstrate stable online behavior and collision-aware motion generation across three scenarios: obstacle-free, static-obstacle, and moving-obstacle environments. The results indicate that the proposed approach generates motion consistent with the operator's command while producing safe detours when obstacles interfere with the commanded path.

Related papers

• Learning Agile Navigation in Crowded Environments for Quadruped Robots [19.29089241882416]
  VOP-Nav is a novel navigation system for quadruped robots.<n>It combines the geometric safety of Velocity Obstacles with the agile adaptability of end-to-end learning.<n>VOP-Nav achieves higher success rates than all baselines while balancing speed and collision avoidance.
  arXiv  Detail & Related papers  (2026-07-16T14:16:19Z)
• Flying Together: Human-Guided Immersive Shared Control for Aerial Robot Teams in Unknown Environments [12.913935671942774]
  We present a Virtual Reality (VR)-based shared control framework for teams of drones operating in constrained and unknown environments.<n>A novel, user-guided motion-primitive-based planner computes continuous, collision-free trajectories while continuously integrating operator input.<n>The system supports mixed-reality operations with both physical and simulated drones, and implements a bilateral VR-based interface.
  arXiv  Detail & Related papers  (2026-05-20T19:37:46Z)
• OmniTrack: General Motion Tracking via Physics-Consistent Reference [54.256312586493415]
  We introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking.<n>Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions.<n>In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels.
  arXiv  Detail & Related papers  (2026-02-27T09:12:15Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Control [62.93681680333618]
  TextOp is a real-time text-driven humanoid motion generation and control framework.<n>It supports streaming language commands and on-the-fly instruction modification during execution.<n>By bridging interactive motion generation with robust whole-body control, TextOp unlocks free-form intent expression.
  arXiv  Detail & Related papers  (2026-02-07T08:42:11Z)
• CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation [70.5382178207975]
  hIsight Perturbation (CHIP) is a plug-and-play module that enables controllable end-effector stiffness.<n>CHIP is easy to implement and requires neither data augmentation nor additional reward tuning.<n>We show that a generalist motion-tracking controller trained with CHIP can perform a diverse set of forceful manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-16T18:56:04Z)
• A Vision-Based Shared-Control Teleoperation Scheme for Controlling the Robotic Arm of a Four-Legged Robot [0.9699673328328621]
  This work proposes an intuitive remote control by leveraging a vision-based pose estimation pipeline.<n>The system maps these wrist movements into robotic arm commands to control the robot's arm in real-time.<n>A trajectory planner ensures safe teleoperation by detecting and preventing collisions with both obstacles and the robotic arm itself.
  arXiv  Detail & Related papers  (2025-08-20T18:31:57Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• Whole-Body Teleoperation for Mobile Manipulation at Zero Added Cost [8.71539730969424]
  MoMa-Teleop is a novel teleoperation method that infers end-effector motions from existing interfaces.<n>We demonstrate that our approach results in a significant reduction in task completion time across a variety of robots and tasks.
  arXiv  Detail & Related papers  (2024-09-23T15:09:45Z)
• Robot Navigation with Entity-Based Collision Avoidance using Deep Reinforcement Learning [0.0]
  We present a novel deep reinforcement learning approach that improves robot navigation and interaction with different types of agents and obstacles.<n>Our approach uses information about the entity types, improving collision avoidance and ensuring safer navigation.<n>We introduce a new reward function that penalizes the robot for being close to or colliding with different entities.
  arXiv  Detail & Related papers  (2024-08-26T11:16:03Z)
• Regularized Deep Signed Distance Fields for Reactive Motion Generation [30.792481441975585]
  Distance-based constraints are fundamental for enabling robots to plan their actions and act safely. We propose Regularized Deep Signed Distance Fields (ReDSDF), a single neural implicit function that can compute smooth distance fields at any scale. We demonstrate the effectiveness of our approach in representative simulated tasks for whole-body control (WBC) and safe Human-Robot Interaction (HRI) in shared workspaces.
  arXiv  Detail & Related papers  (2022-03-09T14:21:32Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.