Fast Bilateral Teleoperation and Imitation Learning Using Sensorless Force Control via Accurate Dynamics Model

• URL: http://arxiv.org/abs/2507.06174v5
• Date: Thu, 24 Jul 2025 00:40:26 GMT
• Title: Fast Bilateral Teleoperation and Imitation Learning Using Sensorless Force Control via Accurate Dynamics Model
• Abstract summary: This work demonstrates that fast teleoperation with force feedback is feasible even with force-sensorless, low-cost manipulators.<n>Based on accurately identified manipulator dynamics, our method integrates nonlinear terms compensation, velocity and external force estimation.<n>Using data collected by 4-channel bilateral control, we show that incorporating force information into both the input and output of learned policies improves performance in imitation learning.
• Score: 1.6019538204169677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, the advancement of imitation learning has led to increased interest in teleoperating low-cost manipulators to collect demonstration data. However, most existing systems rely on unilateral control, which only transmits target position values. While this approach is easy to implement and suitable for slow, non-contact tasks, it struggles with fast or contact-rich operations due to the absence of force feedback. This work demonstrates that fast teleoperation with force feedback is feasible even with force-sensorless, low-cost manipulators by leveraging 4-channel bilateral control. Based on accurately identified manipulator dynamics, our method integrates nonlinear terms compensation, velocity and external force estimation, and variable gain corresponding to inertial variation. Furthermore, using data collected by 4-channel bilateral control, we show that incorporating force information into both the input and output of learned policies improves performance in imitation learning. These results highlight the practical effectiveness of our system for high-fidelity teleoperation and data collection on affordable hardware.

Related papers

• XRoboToolKit-T: Teleoperation with High Stability and Precision with Tactile Sensing for Contact-rich Manipulation [10.905283789732872]
  We propose a versatile teleoperation system with tactile-driven assistance to enable contact-rich manipulation.<n>The proposed XRoboToolKit-T teleoperation system incorporates a tactile-informed force control architecture.<n>The system achieves higher data collection efficiency and improved manipulation stability compared to state-of-the-art teleoperation without assistance.
  arXiv  Detail & Related papers  (2026-09-14T23:24:22Z)
• Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation [17.63466252699545]
  Human-like dexterous hands offer human-level manipulation capabilities but remain difficult to train the control policies that can deploy on real hardware.<n>We present a sim-to-real reinforcement learning method that leverages dense tactile feedback combined with joint torque sensing.<n>This is the first demonstration of controllable grasping on a multi-finger dexterous hand trained entirely in simulation and transferred zero-shot on real hardware.
  arXiv  Detail & Related papers  (2026-07-06T11:15:36Z)
• Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation [51.463992799813816]
  A primary bottleneck in contact-rich manipulation is the difficulty of collecting real-world data.<n>We introduce Center-of-Pressure, an effective tactile representation grounded in physical principles.<n>We evaluate CoP on two blind, challenging contact-rich manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-27T17:59:02Z)
• ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching [53.30290192030814]
  ForceFlow is a force-aware reactive framework built upon flow matching.<n>We show that ForceFlow achieves a 37% success rate improvement over the strong baseline ForceVLA.<n>It also exhibits accurate force signal prediction and superior performance in contact force self-regulation.
  arXiv  Detail & Related papers  (2026-05-11T13:27:00Z)
• Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA [62.16042475700567]
  We introduce IMCopilot, a shared-autonomy assistant to simplify teleoperation data collection.<n>We present MoDE-VLA, an architecture that seamlessly integrates heterogeneous force and tactile modalities into a pretrained VLA backbone.<n>We validate our approach on four tasks of escalating complexity, demonstrating doubled success rate improvement over the baseline in dexterous contact-rich tasks.
  arXiv  Detail & Related papers  (2026-03-09T09:02:30Z)
• SuperSuit: An Isomorphic Bimodal Interface for Scalable Mobile Manipulation [8.367600706539774]
  We present textbfSuperSuit, a bimodal data acquisition framework that supports both robot-in-the-loop teleoperation and active demonstration under a shared kinematic interface.<n>Real-world experiments on long-horizon mobile manipulation tasks show 2.6$times$ higher demonstration throughput in active mode compared to a teleoperation baseline, comparable policy performance when substituting teleoperation data with active demonstrations at fixed dataset size, and monotonic performance improvement as active data volume increases.
  arXiv  Detail & Related papers  (2026-03-06T13:40:30Z)
• TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance [53.35296919674763]
  TouchGuide is a novel cross-policy visuo-tactile fusion paradigm that fuses modalities within a low-dimensional action space.<n>touchGuide operates in two stages to guide a pre-trained diffusion or flow-matching visuomotor policy.<n>To facilitate TouchGuide training with high-quality and cost-effective data, we introduce TacUMI, a data collection system.
  arXiv  Detail & Related papers  (2026-01-28T04:22:47Z)
• Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation [12.509181374985936]
  Human-like dexterous hands with multiple fingers offer human-level manipulation capabilities.<n>But training control policies that can directly deploy on real hardware remains difficult due to contact-rich physics.<n>We present a practical framework that utilizes dense tactile feedback combined with joint torque sensing to regulate physical interactions.
  arXiv  Detail & Related papers  (2026-01-06T07:26:39Z)
• ImplicitRDP: An End-to-End Visual-Force Diffusion Policy with Structural Slow-Fast Learning [52.86018040861575]
  We propose a unified end-to-end visual-force diffusion policy that integrates visual planning and reactive force control within a single network.<n>We introduce Structural Slow-Fast Learning, a mechanism utilizing causal attention to simultaneously process asynchronous visual and force tokens.<n>Experiments on contact-rich tasks demonstrate that ImplicitRDP significantly outperforms both vision-only and hierarchical baselines.
  arXiv  Detail & Related papers  (2025-12-11T18:59:46Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning [16.195474619148793]
  This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs)<n>Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV.<n>We validate our method in various real-world experiments, including full-pose control under load model uncertainties.
  arXiv  Detail & Related papers  (2025-08-02T23:52:33Z)
• Improving Low-Cost Teleoperation: Augmenting GELLO with Force [1.3469274919926264]
  We extend the low-cost GELLO teleoperation system, initially designed for joint position control, with additional force information.<n>Our first extension is to implement force feedback, allowing users to feel resistance when interacting with the environment.<n>Our second extension is to add force information into the data collection process and training of imitation learning models.
  arXiv  Detail & Related papers  (2025-07-18T02:05:07Z)
• Deep Learning Optimization of Two-State Pinching Antennas Systems [48.70043547158868]
  Pinching antennas (PAs) can dynamically control electromagnetic wave propagation through binary activation states.<n>In this work, we investigate the problem of optimally selecting a subset of fixed-position PAs to activate in a waveguide, when the aim is to maximize the communication rate at a user terminal.
  arXiv  Detail & Related papers  (2025-07-08T17:55:54Z)
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation [54.28635581240747]
  Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations.<n>ForceVLA treats external force sensing as a first-class modality within VLA systems.<n>Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control.
  arXiv  Detail & Related papers  (2025-05-28T09:24:25Z)
• Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation [58.95799126311524]
  Humans can accomplish contact-rich tasks using vision and touch, with highly reactive capabilities such as fast response to external changes and adaptive control of contact forces.<n>Existing visual imitation learning approaches rely on action chunking to model complex behaviors.<n>We introduce TactAR, a low-cost teleoperation system that provides real-time tactile feedback through Augmented Reality.
  arXiv  Detail & Related papers  (2025-03-04T18:58:21Z)
• FACTR: Force-Attending Curriculum Training for Contact-Rich Policy Learning [70.65987250853311]
  force feedback is readily available in most robot arms, but not commonly used in teleoperation and policy learning.<n>We present a low-cost, intuitive, bilateral teleoperation setup that relays external forces of the follower arm back to the teacher arm.<n>We then introduce FACTR, a policy learning method that employs a curriculum which corrupts the visual input with decreasing intensity throughout training.
  arXiv  Detail & Related papers  (2025-02-24T18:59:07Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts [61.929388479847525]
  This paper focuses on the problem of detecting and reacting to changes in the distribution of a sensorimotor controller's observables. The key idea is the design of switching policies that can take conformal quantiles as input. We show how to design such policies by using conformal quantiles to switch between base policies with different characteristics.
  arXiv  Detail & Related papers  (2023-11-02T17:59:30Z)
• Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor [14.492202828369127]
  We leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks.<n>We introduce two algorithmic contributions, tactile force matching and learned mode switching, as complimentary methods for improving IL.<n>Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%.
  arXiv  Detail & Related papers  (2023-11-02T14:02:42Z)
• MOMA-Force: Visual-Force Imitation for Real-World Mobile Manipulation [15.333753481333067]
  MOMA-Force is a visual-force imitation method that seamlessly combines representation learning for perception, imitation learning for complex motion generation, and admittance whole-body control for system robustness and controllability. Our method achieves smaller contact forces and smaller force variances compared to baseline methods without force imitation.
  arXiv  Detail & Related papers  (2023-08-07T14:31:07Z)
• Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space [7.116986445066885]
  Reinforcement Learning has led to promising results on a range of challenging decision-making tasks. Fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks. We propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds.
  arXiv  Detail & Related papers  (2021-10-19T12:09:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.