Tri-Manual Visuomotor Imitation Learning of Robot Policies

• URL: http://arxiv.org/abs/2607.25731v3
• Date: Mon, 03 Aug 2026 06:48:34 GMT
• Title: Tri-Manual Visuomotor Imitation Learning of Robot Policies
• Abstract summary: Bimanual teleoperation provides an effective way to collect robot demonstrations.<n>It assumes that the operator and robot have matching numbers of simultaneous control channels.<n>We present TriManPolicy, a tri-manual imitation learning system that allows one operator to demonstrate behaviours for three arms.
• Score: 4.261928070384793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bimanual teleoperation provides an effective way to collect robot demonstrations, but it assumes that the operator and robot have matching numbers of simultaneous control channels. This assumption breaks for tri-manual systems: the robot can coordinate three arms concurrently, whereas a single operator can continuously control only two. Pairwise mode switching may therefore record otherwise independent motions sequentially, causing behaviour cloning to reproduce delays imposed by the interface rather than required by the task. We present TriManPolicy, a tri-manual imitation learning system that allows one operator to demonstrate behaviours for three arms. Its central component is Dependency-Aware Tri-Arm Scheduling (DATS). The key idea is to preserve the demonstrated arm motions while reconsidering when they occur. DATS retimes demonstrations offline by preserving local sensorimotor segments of fixed duration and repositioning them according to constraints on task order and arm usage that are reviewed by a human. The resulting data train a single synchronous policy for all three arms, while deployment requires neither the dependency graph nor the scheduler. Across six challenging tasks performed in the real world, policies trained on demonstrations retimed by DATS exhibit more efficient coordination while maintaining comparable observed task success. Offline analysis further shows that DATS changes the supervision across arms rather than merely removing idle periods. Project videos and additional material are available at https://aus.bot/research/trimanpolicy/.

Related papers

• AHEAD: Anticipatory Hand-Driven Teleoperation via Human Intent Prediction [15.569640550342674]
  AHEAD is a real-time VR teleoperation system that anticipates operator intent to enable proactive, hand-driven control.<n>AHEAD reduces robot reaction latency by 0.6 s (object) and 1.4 s (slot) relative to baselines.
  arXiv  Detail & Related papers  (2026-07-16T16:21:40Z)
• One Demonstration Is Enough for Real-World Robotic Reinforcement Learning [34.23164234669014]
  We present AutoSERL, a framework that leverages a single demonstration to fully automate the intervention process in real-world robot RL.<n>We evaluate AutoSERL on six contact-intensive manipulation tasks across two robot platforms, spanning insertion, hanging, and hinge-based tasks.
  arXiv  Detail & Related papers  (2026-07-02T03:23:40Z)
• Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies [8.640835924450553]
  Collaborative mobile manipulation requires robots to coordinate with a partially observed partner.<n>This is difficult because failures often arise from mistimed waiting, yielding, pulling, releasing, or repositioning.<n>We propose Sequential Asymmetric Imitation (SAI), a single-teleoperator curriculum for learning coupled multi-robot behaviors.
  arXiv  Detail & Related papers  (2026-06-15T09:55:58Z)
• HECTOR: Human-centric Hierarchical Coordination and Supervision of Robotic Fleets under Continual Temporal Tasks [7.155003467581939]
  This work proposes a human-centric coordination and supervision scheme (HECTOR) for large-scale robotic fleets under continual and uncertain temporal tasks.<n>It consists of three hierarchical layers: (I) the bidirectional and multimodal protocol of online human-fleet interaction, where the operator interacts with and supervises the whole fleet; (II) the rolling assignment of currently-known tasks to teams within a certain horizon, and (III) the dynamic coordination within a team given the detected subtasks during online execution.
  arXiv  Detail & Related papers  (2026-04-13T01:39:19Z)
• CORAL: Scalable Multi-Task Robot Learning via LoRA Experts [49.759823970016974]
  Vision-Language-Action (VLA) models in real-world robotics expose a core multi-task learning challenge.<n>Maintaining a separate full checkpoint per task is often storage- and deployment-prohibitive.<n>We present CORAL, a backbone- and-agnostic framework designed primarily to mitigate multi-task interference.
  arXiv  Detail & Related papers  (2026-03-10T07:28:41Z)
• CARE: Multi-Task Pretraining for Latent Continuous Action Representation in Robot Control [39.17038025776311]
  CARE is a framework designed to train VLA models for robotic task execution.<n> CARE eliminates the need for explicit action labels by leveraging only video-text pairs.<n>Results demonstrate CARE's scalability, interpretability, and effectiveness in robotic control with weak supervision.
  arXiv  Detail & Related papers  (2026-01-30T02:28:32Z)
• CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation [70.5382178207975]
  hIsight Perturbation (CHIP) is a plug-and-play module that enables controllable end-effector stiffness.<n>CHIP is easy to implement and requires neither data augmentation nor additional reward tuning.<n>We show that a generalist motion-tracking controller trained with CHIP can perform a diverse set of forceful manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-16T18:56:04Z)
• Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models [49.4824734958566]
  Chain-of-Modality (CoM) enables Vision Language Models to reason about multimodal human demonstration data.<n>CoM refines a task plan and generates detailed control parameters, enabling robots to perform manipulation tasks based on a single multimodal human video prompt.
  arXiv  Detail & Related papers  (2025-04-17T21:31:23Z)
• Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework [51.39847596489193]
  Bimanual robotic manipulation is an emerging and critical topic in the robotics community.<n>We propose a novel decoupled interaction framework that considers the characteristics of different tasks in bimanual manipulation.<n>Our framework achieves outstanding performance, with a 23.5% boost over the SOTA method.
  arXiv  Detail & Related papers  (2025-03-12T09:28:41Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up.<n>It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention.<n>It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• Whole-Body Teleoperation for Mobile Manipulation at Zero Added Cost [8.71539730969424]
  MoMa-Teleop is a novel teleoperation method that infers end-effector motions from existing interfaces.<n>We demonstrate that our approach results in a significant reduction in task completion time across a variety of robots and tasks.
  arXiv  Detail & Related papers  (2024-09-23T15:09:45Z)
• ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation [31.211870350260703]
  Keypoint Constraints (ReKep) is a visually-grounded representation for constraints in robotic manipulation. ReKep is expressed as Python functions mapping a set of 3D keypoints the environment to a numerical cost. We present system implementations on a wheeled single-arm platform and a stationary dual-arm platform.
  arXiv  Detail & Related papers  (2024-09-03T06:45:22Z)
• Learning Multi-Arm Manipulation Through Collaborative Teleoperation [63.35924708783826]
  Imitation Learning (IL) is a powerful paradigm to teach robots to perform manipulation tasks. Many real-world tasks require multiple arms, such as lifting a heavy object or assembling a desk. We present Multi-Arm RoboTurk (MART), a multi-user data collection platform that allows multiple remote users to simultaneously teleoperate a set of robotic arms.
  arXiv  Detail & Related papers  (2020-12-12T05:43:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.