CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion

• URL: http://arxiv.org/abs/2211.00458v1
• Date: Tue, 1 Nov 2022 13:41:13 GMT
• Title: CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion
• Authors: Guillaume Bellegarda, Auke Ijspeert
• Abstract summary: We present a method for integrating central pattern generators (CPGs) into the deep reinforcement learning framework to produce robust quadruped locomotion. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training.
• Score: 4.56877715768796
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training, most notably to a dynamically added 13.75 kg load representing 115% of the nominal quadruped mass. We test several different observation spaces based on proprioceptive sensing and show that our framework is deployable with no domain randomization and very little feedback, where along with the oscillator states, it is possible to provide only contact booleans in the observation space. Video results can be found at https://youtu.be/xqXHLzLsEV4.

Related papers

• Flexible Gravitational-Wave Parameter Estimation with Transformers [73.44614054040267]
  We introduce a flexible transformer-based architecture paired with a training strategy that enables adaptation to diverse analysis settings at inference time.<n>We demonstrate that a single flexible model -- called Dingo-T1 -- can analyze 48 gravitational-wave events from the third LIGO-Virgo-KAGRA Observing Run.
  arXiv  Detail & Related papers  (2025-12-02T17:49:08Z)
• From Black Hole to Galaxy: Neural Operator: Framework for Accretion and Feedback Dynamics [70.27068115318681]
  We introduce a neural-based ''subgrid black hole'' that learns the small-scale local dynamics and embeds it within direct simulations.<n>Thanks to the great speedup in fine-scale evolution, our approach captures intrinsic variability in accretion-driven feedback, allowing dynamic coupling between the central black hole and galaxy-scale gas.
  arXiv  Detail & Related papers  (2025-12-01T11:47:49Z)
• RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking [50.200035833530876]
  RobotDancing is a simple, scalable framework that predicts residual joint targets to explicitly correct dynamics discrepancies.<n>It can track multi-minute, high-energy behaviors (jumps, spins, cartwheels) and deploys zero-shot to hardware with high motion tracking quality.
  arXiv  Detail & Related papers  (2025-09-25T03:30:34Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Learning Emergent Gaits with Decentralized Phase Oscillators: on the role of Observations, Rewards, and Feedback [16.290816894141003]
  We present a minimal phase oscillator model for learning quadrupedal locomotion. We show that the combination of phase observations, simple phase-based rewards, and the local feedback dynamics induces policies that exhibit emergent gait preferences.
  arXiv  Detail & Related papers  (2024-02-13T18:46:10Z)
• Inferring Relational Potentials in Interacting Systems [56.498417950856904]
  We propose Neural Interaction Inference with Potentials (NIIP) as an alternative approach to discover such interactions. NIIP assigns low energy to the subset of trajectories which respect the relational constraints observed. It allows trajectory manipulation, such as interchanging interaction types across separately trained models, as well as trajectory forecasting.
  arXiv  Detail & Related papers  (2023-10-23T00:44:17Z)
• Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control [63.310780486820796]
  We show how a parameterization of recurrent connectivity influences robustness in closed-loop settings. We find that closed-form continuous-time neural networks (CfCs) with fewer parameters can outperform their full-rank, fully-connected counterparts.
  arXiv  Detail & Related papers  (2023-10-05T21:44:18Z)
• Generating Oscillation Activity with Echo State Network to Mimic the Behavior of a Simple Central Pattern Generator [13.021014899410684]
  This paper presents a method for reproducing a simple central pattern generator (CPG) using a modified Echo State Network (ESN) We define the specific neuron ensemble required for generating oscillations in the reservoir and demonstrate how adjustments to the leaking rate, spectral radius, topology, and population size can increase the probability of reproducing these oscillations. This approach offers a promising solution for the development of bio-inspired controllers for robotic systems.
  arXiv  Detail & Related papers  (2023-06-19T13:37:12Z)
• Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion [4.557963624437784]
  We present a framework for learning visually-guided quadruped locomotion. We integrate exteroceptive sensing and central pattern generators into the deep reinforcement learning framework. Our results show that the CPG, explicit interoscillator couplings, and memory-enabled policy representations are all beneficial for energy efficiency.
  arXiv  Detail & Related papers  (2022-12-29T18:14:38Z)
• Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion [10.838285018473725]
  We demonstrate robust and dynamic locomotion with a learned motion controller executing at as low as 8 Hz on a real ANYmal C quadruped. The robot is able to robustly and repeatably achieve a high heading velocity of 1.5 m/s, traverse uneven terrain, and resist unexpected external perturbations.
  arXiv  Detail & Related papers  (2022-09-29T15:55:33Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)
• Gate-based spin readout of hole quantum dots with site-dependent $g-$factors [101.23523361398418]
  We experimentally investigate a hole double quantum dot in silicon by carrying out spin readout with gate-based reflectometry. We show that characteristic features in the reflected phase signal arising from magneto-spectroscopy convey information on site-dependent $g-$factors in the two dots.
  arXiv  Detail & Related papers  (2022-06-27T09:07:20Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• Neuromorphic adaptive spiking CPG towards bio-inspired locomotion of legged robots [58.720142291102135]
  Spiking Central Pattern Generator generates different locomotion patterns driven by an external stimulus. The locomotion of the end robotic platform (any-legged robot) can be adapted to the terrain by using any sensor as input.
  arXiv  Detail & Related papers  (2021-01-24T12:44:38Z)
• Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot [19.80726424244039]
  We present a new locomotion control method for soft robot snakes inspired by biological snakes. The performance of the proposed controller is experimentally validated with both simulated and real soft snake robots.
  arXiv  Detail & Related papers  (2020-01-13T04:32:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.