X-OP: Cross-Morphology Whole-Body Teleoperation via MPC Retargeting

• URL: http://arxiv.org/abs/2606.07934v1
• Date: Sat, 06 Jun 2026 01:50:59 GMT
• Title: X-OP: Cross-Morphology Whole-Body Teleoperation via MPC Retargeting
• Abstract summary: Whole-body teleoperation is essential for scalable robot data collection in loco-manipulation tasks.<n>We propose a hierarchical whole-body teleoperation framework driven by a single XR device.<n>We show higher success rates on whole-body control tasks for both a humanoid and a mobile manipulator.
• Score: 3.134050607080983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Whole-body teleoperation is essential for scalable robot data collection in loco-manipulation tasks, yet existing approaches relying on exoskeleton suits or multi-camera setups impose prohibitive cost, complexity, and environmental constraints. Recent methods using a single extended reality (XR) device with end-to-end reinforcement learning policies partially address these limitations but require robot-specific retraining, suffer from out-of-distribution failures, and rely on motion retargeting that neglects dynamic feasibility. We propose a hierarchical whole-body teleoperation framework driven by a single XR device that generalizes across diverse robot morphologies without retraining robot-specific policies. A Model Predictive Control (MPC)-based motion retargeter jointly optimizes alignment with the operator's intent and the robot's dynamic feasibility, generating optimal commands for existing low-level controllers. To ensure robust online execution, we introduce a state synchronization method that resets the simulator state at each MPC step to handle noisy real-world measurements and contact sensitivity, and integrate SLAM-based global pose feedback to mitigate long-term drift. Simulation results show higher success rates on whole-body control tasks for both a humanoid (over 30% lower completion time and 20% lower power consumption) and a mobile manipulator (zero collisions) compared to baselines. Real-world experiments further validate the effectiveness and flexibility of our method, demonstrating the successful deployment of the proposed retargeter on both platforms for whole-body control tasks and the ease of allowing users to adjust teleoperation behavior based on their preferences. This plug-and-play framework offers a scalable, morphology-agnostic solution for whole-body robot teleoperation, enabling real-time behavioral customization and broad applicability across platforms.

Related papers

• GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch [66.7936140772745]
  We present GigaWorld-Policy-0.5, an action-centered WAM designed for more efficient robot control.<n>For efficient inference, GigaWorld-Policy-0.5 introduces a Mixture-of-Transformers architecture that separates visual dynamics modeling and action generation into specialized experts.<n>Experiments and ablations show that GigaWorld-Policy-0.5 preserves the training benefits of future visual dynamics while improving inference efficiency for robot control.
  arXiv  Detail & Related papers  (2026-07-15T15:46:42Z)
• Domain-Adaptive Communication-Rate Optimization for Sim-to-Real Humanoid-Robot Wireless XR Teleoperation [45.11184664928145]
  Wireless extended reality (XR) teleoperation provides embodied interaction capability for collecting humanoid robot demonstrations.<n>This paper develops a system framework that integrates sampling, transmission, the reconstruction and formulates a communication-rate optimization.
  arXiv  Detail & Related papers  (2026-05-19T03:13:09Z)
• ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation [66.02142169323521]
  Vision-Language-Action models and world models have recently emerged as promising paradigms for general-purpose robotic intelligence.<n>Existing benchmarks are largely simulator-centric, which provide controllability but fail to capture the reality gap caused by perception noise.<n>We introduce ManipArena, a standardized evaluation framework designed to bridge simulation and real-world execution.
  arXiv  Detail & Related papers  (2026-03-30T15:06:41Z)
• OmniClone: Engineering a Robust, All-Rounder Whole-Body Humanoid Teleoperation System [39.72804196709836]
  Whole-body humanoid teleoperation enables humans to remotely control humanoid robots, serving as both a real-time operational tool and a scalable engine for collecting demonstrations for autonomous learning.<n>Here we present OmniClone, a whole-body humanoid teleoperation system that achieves high-fidelity, multi-skill control on a single consumer with modest data requirements.
  arXiv  Detail & Related papers  (2026-03-15T11:13:04Z)
• ACLM: ADMM-Based Distributed Model Predictive Control for Collaborative Loco-Manipulation [9.708461585583791]
  Collaborative transportation of heavy payloads via loco-manipulation is a challenging yet essential capability for legged robots.<n>This study proposes an Alternating Direction Method of Multipliers-based distributed model predictive control framework for loco-manipulation.
  arXiv  Detail & Related papers  (2026-03-07T08:06:51Z)
• RobotArena $\infty$: Scalable Robot Benchmarking via Real-to-Sim Translation [47.79800816696372]
  Real-world testing of manipulation policies is labor-intensive at scale, and difficult to reproduce.<n>Existing simulation benchmarks are similarly limited, as they train and test policies within the same synthetic domains.<n>In this paper, we introduce a new benchmarking framework that overcomes these challenges by shifting VLA evaluation into large-scale simulated augmented environments.
  arXiv  Detail & Related papers  (2025-10-27T17:41:38Z)
• Ctrl-World: A Controllable Generative World Model for Robot Manipulation [53.71061464925014]
  Generalist robot policies can perform a wide range of manipulation skills.<n> evaluating and improving their ability with unfamiliar objects and instructions remains a significant challenge.<n>World models offer a promising, scalable alternative by enabling policies to rollout within imagination space.
  arXiv  Detail & Related papers  (2025-10-11T09:13:10Z)
• OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction [76.44108003274955]
  A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning policies.<n>We introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh.<n>By minimizing the Laplacian deformation between the human and robot meshes, OmniRetarget generates kinematically feasible trajectories.
  arXiv  Detail & Related papers  (2025-09-30T17:59:02Z)
• Feedback-MPPI: Fast Sampling-Based MPC via Rollout Differentiation -- Adios low-level controllers [0.9674641730446749]
  Model Predictive Path Integral control is a powerful sampling-based approach suitable for complex robotic tasks.<n>This paper introduces robust feedback gains derived from sensitivity used in gradient-based MPC.<n>We demonstrate the effectiveness of F-MPPI in simulations through real-world experiments on two robotic platforms.
  arXiv  Detail & Related papers  (2025-06-17T07:47:33Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.