Fast Bilateral Teleoperation and Imitation Learning Using Sensorless Force Control via Accurate Dynamics Model

• URL: http://arxiv.org/abs/2507.06174v5
• Date: Thu, 24 Jul 2025 00:40:26 GMT
• Title: Fast Bilateral Teleoperation and Imitation Learning Using Sensorless Force Control via Accurate Dynamics Model
• Authors: Koki Yamane, Yunhan Li, Masashi Konosu, Koki Inami, Junji Oaki, Sho Sakaino, Toshiaki Tsuji,
• Abstract summary: This work demonstrates that fast teleoperation with force feedback is feasible even with force-sensorless, low-cost manipulators.<n>Based on accurately identified manipulator dynamics, our method integrates nonlinear terms compensation, velocity and external force estimation.<n>Using data collected by 4-channel bilateral control, we show that incorporating force information into both the input and output of learned policies improves performance in imitation learning.
• Score: 1.6019538204169677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, the advancement of imitation learning has led to increased interest in teleoperating low-cost manipulators to collect demonstration data. However, most existing systems rely on unilateral control, which only transmits target position values. While this approach is easy to implement and suitable for slow, non-contact tasks, it struggles with fast or contact-rich operations due to the absence of force feedback. This work demonstrates that fast teleoperation with force feedback is feasible even with force-sensorless, low-cost manipulators by leveraging 4-channel bilateral control. Based on accurately identified manipulator dynamics, our method integrates nonlinear terms compensation, velocity and external force estimation, and variable gain corresponding to inertial variation. Furthermore, using data collected by 4-channel bilateral control, we show that incorporating force information into both the input and output of learned policies improves performance in imitation learning. These results highlight the practical effectiveness of our system for high-fidelity teleoperation and data collection on affordable hardware.

Related papers

• Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning [16.195474619148793]
  This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs)<n>Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV.<n>We validate our method in various real-world experiments, including full-pose control under load model uncertainties.
  arXiv  Detail & Related papers  (2025-08-02T23:52:33Z)
• Improving Low-Cost Teleoperation: Augmenting GELLO with Force [1.3469274919926264]
  We extend the low-cost GELLO teleoperation system, initially designed for joint position control, with additional force information.<n>Our first extension is to implement force feedback, allowing users to feel resistance when interacting with the environment.<n>Our second extension is to add force information into the data collection process and training of imitation learning models.
  arXiv  Detail & Related papers  (2025-07-18T02:05:07Z)
• Deep Learning Optimization of Two-State Pinching Antennas Systems [48.70043547158868]
  Pinching antennas (PAs) can dynamically control electromagnetic wave propagation through binary activation states.<n>In this work, we investigate the problem of optimally selecting a subset of fixed-position PAs to activate in a waveguide, when the aim is to maximize the communication rate at a user terminal.
  arXiv  Detail & Related papers  (2025-07-08T17:55:54Z)
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation [54.28635581240747]
  Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations.<n>ForceVLA treats external force sensing as a first-class modality within VLA systems.<n>Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control.
  arXiv  Detail & Related papers  (2025-05-28T09:24:25Z)
• Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation [58.95799126311524]
  Humans can accomplish contact-rich tasks using vision and touch, with highly reactive capabilities such as fast response to external changes and adaptive control of contact forces.<n>Existing visual imitation learning approaches rely on action chunking to model complex behaviors.<n>We introduce TactAR, a low-cost teleoperation system that provides real-time tactile feedback through Augmented Reality.
  arXiv  Detail & Related papers  (2025-03-04T18:58:21Z)
• FACTR: Force-Attending Curriculum Training for Contact-Rich Policy Learning [70.65987250853311]
  force feedback is readily available in most robot arms, but not commonly used in teleoperation and policy learning.<n>We present a low-cost, intuitive, bilateral teleoperation setup that relays external forces of the follower arm back to the teacher arm.<n>We then introduce FACTR, a policy learning method that employs a curriculum which corrupts the visual input with decreasing intensity throughout training.
  arXiv  Detail & Related papers  (2025-02-24T18:59:07Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts [61.929388479847525]
  This paper focuses on the problem of detecting and reacting to changes in the distribution of a sensorimotor controller's observables. The key idea is the design of switching policies that can take conformal quantiles as input. We show how to design such policies by using conformal quantiles to switch between base policies with different characteristics.
  arXiv  Detail & Related papers  (2023-11-02T17:59:30Z)
• Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor [14.492202828369127]
  We leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks.<n>We introduce two algorithmic contributions, tactile force matching and learned mode switching, as complimentary methods for improving IL.<n>Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%.
  arXiv  Detail & Related papers  (2023-11-02T14:02:42Z)
• MOMA-Force: Visual-Force Imitation for Real-World Mobile Manipulation [15.333753481333067]
  MOMA-Force is a visual-force imitation method that seamlessly combines representation learning for perception, imitation learning for complex motion generation, and admittance whole-body control for system robustness and controllability. Our method achieves smaller contact forces and smaller force variances compared to baseline methods without force imitation.
  arXiv  Detail & Related papers  (2023-08-07T14:31:07Z)
• Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space [7.116986445066885]
  Reinforcement Learning has led to promising results on a range of challenging decision-making tasks. Fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks. We propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds.
  arXiv  Detail & Related papers  (2021-10-19T12:09:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.