Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation

• URL: http://arxiv.org/abs/2502.10894v1
• Date: Sat, 15 Feb 2025 20:18:37 GMT
• Title: Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation
• Authors: Nolan Fey, Gabriel B. Margolis, Martin Peticco, Pulkit Agrawal,
• Abstract summary: We introduce the Unsupervised Actuator Net (UAN) to bridge the sim-to-real gap for complex actuation mechanisms.<n>UAN mitigates reward hacking by ensuring that the learned behaviors remain robust and transferable.<n>With these innovations, our robot athlete learns to lift, throw, and drag with remarkable fidelity from simulation to reality.
• Score: 18.451995260533682
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Achieving athletic loco-manipulation on robots requires moving beyond traditional tracking rewards - which simply guide the robot along a reference trajectory - to task rewards that drive truly dynamic, goal-oriented behaviors. Commands such as "throw the ball as far as you can" or "lift the weight as quickly as possible" compel the robot to exhibit the agility and power inherent in athletic performance. However, training solely with task rewards introduces two major challenges: these rewards are prone to exploitation (reward hacking), and the exploration process can lack sufficient direction. To address these issues, we propose a two-stage training pipeline. First, we introduce the Unsupervised Actuator Net (UAN), which leverages real-world data to bridge the sim-to-real gap for complex actuation mechanisms without requiring access to torque sensing. UAN mitigates reward hacking by ensuring that the learned behaviors remain robust and transferable. Second, we use a pre-training and fine-tuning strategy that leverages reference trajectories as initial hints to guide exploration. With these innovations, our robot athlete learns to lift, throw, and drag with remarkable fidelity from simulation to reality.

Related papers

• Training Directional Locomotion for Quadrupedal Low-Cost Robotic Systems via Deep Reinforcement Learning [4.669957449088593]
  We present Deep Reinforcement Learning training of directional locomotion for low-cost quadpedalru robots in the real world. We exploit randomization of heading that the robot must follow to foster exploration of action-state transitions. Changing the heading in episode resets to current yaw plus a random value drawn from a normal distribution yields policies able to follow complex trajectories.
  arXiv  Detail & Related papers  (2025-03-14T03:53:01Z)
• Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning [54.26816599309778]
  We propose a novel whole-body locomotion algorithm based on dynamic balance and Reinforcement Learning (RL) Specifically, we introduce a dynamic balance mechanism by leveraging an extended measure of Zero-Moment Point (ZMP)-driven rewards and task-driven rewards in a whole-body actor-critic framework. Experiments conducted on a full-sized Unitree H1-2 robot verify the ability of our method to maintain balance on extremely narrow terrains.
  arXiv  Detail & Related papers  (2025-02-24T14:53:45Z)
• Moto: Latent Motion Token as the Bridging Language for Robot Manipulation [66.18557528695924]
  We introduce Moto, which converts video content into latent Motion Token sequences by a Latent Motion Tokenizer.<n>We pre-train Moto-GPT through motion token autoregression, enabling it to capture diverse visual motion knowledge.<n>To transfer learned motion priors to real robot actions, we implement a co-fine-tuning strategy that seamlessly bridges latent motion token prediction and real robot control.
  arXiv  Detail & Related papers  (2024-12-05T18:57:04Z)
• DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision [26.9579556496875]
  Joint manipulation of moving objects and locomotion with legs, such as playing soccer, receive scant attention in the learning community. We propose a feedback control block to compute the necessary body-level movement accurately and using the outputs as dynamic joint-level locomotion supervision. We observe that our learning scheme can not only make the policy network converge faster but also enable soccer robots to perform sophisticated maneuvers.
  arXiv  Detail & Related papers  (2024-03-21T11:16:28Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Barkour: Benchmarking Animal-level Agility with Quadruped Robots [70.97471756305463]
  We introduce the Barkour benchmark, an obstacle course to quantify agility for legged robots. Inspired by dog agility competitions, it consists of diverse obstacles and a time based scoring mechanism. We present two methods for tackling the benchmark.
  arXiv  Detail & Related papers  (2023-05-24T02:49:43Z)
• Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning [26.13655448415553]
  Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. The resulting agent exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more.
  arXiv  Detail & Related papers  (2023-04-26T16:25:54Z)
• Learning and Adapting Agile Locomotion Skills by Transferring Experience [71.8926510772552]
  We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments.
  arXiv  Detail & Related papers  (2023-04-19T17:37:54Z)
• Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion [34.33972863987201]
  We train quadruped robots to use the front legs to climb walls, press buttons, and perform object interaction in the real world. These skills are trained in simulation using curriculum and transferred to the real world using our proposed sim2real variant. We evaluate our method in both simulation and real-world showing successful executions of both short as well as long-range tasks.
  arXiv  Detail & Related papers  (2023-03-20T17:59:58Z)
• GraspARL: Dynamic Grasping via Adversarial Reinforcement Learning [16.03016392075486]
  We introduce an adversarial reinforcement learning framework for dynamic grasping, namely GraspARL. We formulate the dynamic grasping problem as a'move-and-grasp' game, where the robot is to pick up the object on the mover and the adversarial mover is to find a path to escape it. In this way, the mover can auto-generate diverse moving trajectories while training. And the robot trained with the adversarial trajectories can generalize to various motion patterns.
  arXiv  Detail & Related papers  (2022-03-04T03:25:09Z)
• Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots [121.42930679076574]
  We present a model-free reinforcement learning framework for training robust locomotion policies in simulation. domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw.
  arXiv  Detail & Related papers  (2021-03-26T07:14:01Z)
• Learning Agile Locomotion via Adversarial Training [59.03007947334165]
  In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effectively alleviates the laborious environment design effort. In contrast to prior works that used only one adversary, we find that training an ensemble of adversaries, each of which specializes in a different escaping strategy, is essential for the protagonist to master agility.
  arXiv  Detail & Related papers  (2020-08-03T01:20:37Z)
• Learning to Play Table Tennis From Scratch using Muscular Robots [34.34824536814943]
  This work is the first to (a) fail-safe learn of a safety-critical dynamic task using anthropomorphic robot arms, (b) learn a precision-demanding problem with a PAM-driven system, and (c) train robots to play table tennis without real balls. Videos and datasets are available at muscularTT.embodied.ml.
  arXiv  Detail & Related papers  (2020-06-10T16:43:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.