Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion

• URL: http://arxiv.org/abs/2212.14400v2
• Date: Mon, 11 Mar 2024 16:49:16 GMT
• Title: Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion
• Authors: Guillaume Bellegarda, Milad Shafiee, Auke Ijspeert
• Abstract summary: We present a framework for learning visually-guided quadruped locomotion. We integrate exteroceptive sensing and central pattern generators into the deep reinforcement learning framework. Our results show that the CPG, explicit interoscillator couplings, and memory-enabled policy representations are all beneficial for energy efficiency.
• Score: 4.557963624437784
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a framework for learning visually-guided quadruped locomotion by integrating exteroceptive sensing and central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework. Through both exteroceptive and proprioceptive sensing, the agent learns to coordinate rhythmic behavior among different oscillators to track velocity commands, while at the same time override these commands to avoid collisions with the environment. We investigate several open robotics and neuroscience questions: 1) What is the role of explicit interoscillator couplings between oscillators, and can such coupling improve sim-to-real transfer for navigation robustness? 2) What are the effects of using a memory-enabled vs. a memory-free policy network with respect to robustness, energy-efficiency, and tracking performance in sim-to-real navigation tasks? 3) How do animals manage to tolerate high sensorimotor delays, yet still produce smooth and robust gaits? To answer these questions, we train our perceptive locomotion policies in simulation and perform sim-to-real transfers to the Unitree Go1 quadruped, where we observe robust navigation in a variety of scenarios. Our results show that the CPG, explicit interoscillator couplings, and memory-enabled policy representations are all beneficial for energy efficiency, robustness to noise and sensory delays of 90 ms, and tracking performance for successful sim-to-real transfer for navigation tasks. Video results can be found at https://youtu.be/wpsbSMzIwgM.

Related papers

• MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning [66.53533434848369]
  We propose a motion-guided self-learning framework that learns densely consistent representations.<n>We improve state-of-the-art by 1% to 6% on six image and video datasets and four evaluation benchmarks.
  arXiv  Detail & Related papers  (2025-06-10T11:20:32Z)
• PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers [10.598333486002]
  Humans excel in navigating diverse, complex environments with agile motor skills, exemplified by parkour practitioners performing dynamic maneuvers.<n>Reproducing these agile movements with simulated characters remains challenging due to the scarcity of motion capture data.<n>We introduce PARC, a framework that leverages machine learning and physics-based simulation to iteratively augment motion datasets.
  arXiv  Detail & Related papers  (2025-05-06T22:29:07Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up. It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention. It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion [4.56877715768796]
  We present a method for integrating central pattern generators (CPGs) into the deep reinforcement learning framework to produce robust quadruped locomotion. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training.
  arXiv  Detail & Related papers  (2022-11-01T13:41:13Z)
• Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion [10.838285018473725]
  We demonstrate robust and dynamic locomotion with a learned motion controller executing at as low as 8 Hz on a real ANYmal C quadruped. The robot is able to robustly and repeatably achieve a high heading velocity of 1.5 m/s, traverse uneven terrain, and resist unexpected external perturbations.
  arXiv  Detail & Related papers  (2022-09-29T15:55:33Z)
• Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments [25.366480092589022]
  A quadrupedal robot must exhibit robust and agile walking behaviors in response to environmental clutter and moving obstacles. We present a hierarchical learning framework, named PRELUDE, which decomposes the problem of perceptive locomotion into high-level decision-making. We demonstrate the effectiveness of our approach in simulation and with hardware experiments.
  arXiv  Detail & Related papers  (2022-09-19T17:55:07Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking [72.65494220685525]
  We propose a new dynamic modality-aware filter generation module (named MFGNet) to boost the message communication between visible and thermal data. We generate dynamic modality-aware filters with two independent networks. The visible and thermal filters will be used to conduct a dynamic convolutional operation on their corresponding input feature maps respectively. To address issues caused by heavy occlusion, fast motion, and out-of-view, we propose to conduct a joint local and global search by exploiting a new direction-aware target-driven attention mechanism.
  arXiv  Detail & Related papers  (2021-07-22T03:10:51Z)
• DriveGAN: Towards a Controllable High-Quality Neural Simulation [147.6822288981004]
  We introduce a novel high-quality neural simulator referred to as DriveGAN. DriveGAN achieves controllability by disentangling different components without supervision. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data.
  arXiv  Detail & Related papers  (2021-04-30T15:30:05Z)
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  LiDAR-based panoptic segmentation aims to parse both objects and scenes in a unified manner. We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-24T08:44:46Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)
• CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion [0.0]
  We present CARL, a quadruped agent that can be controlled with high-level directives and react naturally to dynamic environments. We use Generative Adrial Networks to adapt high-level controls, such as speed and heading, to action distributions that correspond to the original animations. Further fine-tuning through the deep reinforcement learning enables the agent to recover from unseen external perturbations while producing smooth transitions.
  arXiv  Detail & Related papers  (2020-05-07T07:18:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.