Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2407.01050v1
• Date: Mon, 1 Jul 2024 07:57:01 GMT
• Title: Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning
• Abstract summary: This paper investigates a novel class of overconstrained locomotion from a design and learning perspective inspired by evolutionary morphology. We propose a 3D-printable design of robotic limbs parametrically reconfigurable as a classical planar 4-bar linkage, an overconstrained Bennett linkage, and a spherical 4-bar linkage. Results show that the overconstrained limbs exhibit more efficient locomotion than planar limbs during forward and sideways walking over different terrains.
• Score: 10.030349447312057
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: While the animals' Fin-to-Limb evolution has been well-researched in biology, such morphological transformation remains under-adopted in the modern design of advanced robotic limbs. This paper investigates a novel class of overconstrained locomotion from a design and learning perspective inspired by evolutionary morphology, aiming to integrate the concept of `intelligent design under constraints' - hereafter referred to as constraint-driven design intelligence - in developing modern robotic limbs with superior energy efficiency. We propose a 3D-printable design of robotic limbs parametrically reconfigurable as a classical planar 4-bar linkage, an overconstrained Bennett linkage, and a spherical 4-bar linkage. These limbs adopt a co-axial actuation, identical to the modern legged robot platforms, with the added capability of upgrading into a wheel-legged system. Then, we implemented a large-scale, multi-terrain deep reinforcement learning framework to train these reconfigurable limbs for a comparative analysis of overconstrained locomotion in energy efficiency. Results show that the overconstrained limbs exhibit more efficient locomotion than planar limbs during forward and sideways walking over different terrains, including floors, slopes, and stairs, with or without random noises, by saving at least 22% mechanical energy in completing the traverse task, with the spherical limbs being the least efficient. It also achieves the highest average speed of 0.85 meters per second on flat terrain, which is 20% faster than the planar limbs. This study paves the path for an exciting direction for future research in overconstrained robotics leveraging evolutionary morphology and reconfigurable mechanism intelligence when combined with state-of-the-art methods in deep reinforcement learning.

Related papers

• Diff-Muscle: Efficient Learning for Musculoskeletal Robotic Table Tennis [21.901628569185515]
  We propose Diff-Muscle, a musculoskeletal robot control algorithm that reformulates policy learning from the redundant muscle-activation space into a significantly lower-dimensional joint space.<n>Specifically, we propose a hierarchical reinforcement learning framework that integrates a Kinematics-based Muscle Actuation Controller (K-MAC) with high-level trajectory planning.<n> Experimental results demonstrate that Diff-Muscle significantly outperforms state-of-the-art baselines in success rates while maintaining minimal muscle activation.
  arXiv  Detail & Related papers  (2026-03-09T17:00:28Z)
• Terrain characterization and locomotion adaptation in a small-scale lizard-inspired robot [0.2770822269241973]
  We develop a Small-scale, Intelligent, Lizard-inspired, Adaptive Robot (SILA Bot) capable of adapting to diverse substrates.<n>We show that the optimal body movement pattern can be parameterized and approximated as a linear function of granular depth.<n>We also design a simple linear feedback controller that modulates body phase and substantially improves performance on terrains with unknown depth.
  arXiv  Detail & Related papers  (2026-03-06T02:44:59Z)
• Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching [77.28042137892943]
  We present Perceptive Humanoid Parkour (PHP), a modular framework that enables humanoid robots to autonomously perform long-horizon, vision-based parkour.<n>We train motion-tracking reinforcement learning expert policies for these composed motions, and distill them into a single depth-based, multi-skill student policy.<n>We validate our framework with extensive real-world experiments on a Unitree G1 humanoid robot.
  arXiv  Detail & Related papers  (2026-02-17T18:59:11Z)
• Agile asymmetric multi-legged locomotion: contact planning via geometric mechanics and spin model duality [15.47009589379934]
  We develop a principled framework for discovering new control structures in multi-legged locomotion.<n>We use geometric mechanics to reduce contact-rich locomotion planning to a graph optimization problem.<n>We identify an asymmetric locomotion strategy for a hexapod robot that achieves a forward speed of 0.61 body lengths per cycle.
  arXiv  Detail & Related papers  (2026-02-09T19:13:14Z)
• Toward Humanoid Brain-Body Co-design: Joint Optimization of Control and Morphology for Fall Recovery [55.951691393378354]
  We propose RoboCraft, a scalable humanoid co-design framework for fall recovery.<n>A shared policy pretrained across multiple designs is progressively finetuned on high-performing morphologies.<n>Experiments show that RoboCraft achieves an average performance gain of 44.55% on seven public humanoid robots.
  arXiv  Detail & Related papers  (2025-10-25T15:40:18Z)
• PALo: Learning Posture-Aware Locomotion for Quadruped Robots [29.582249837902427]
  We propose an end-to-end deep reinforcement learning framework for posture-aware locomotion named PALo. PALo handles simultaneous linear and angular velocity tracking and real-time adjustments of body height, pitch, and roll angles. PALo achieves agile posture-aware locomotion control in simulated environments and successfully transfers to real-world settings without fine-tuning.
  arXiv  Detail & Related papers  (2025-03-06T14:13:59Z)
• Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning [54.26816599309778]
  We propose a novel whole-body locomotion algorithm based on dynamic balance and Reinforcement Learning (RL) Specifically, we introduce a dynamic balance mechanism by leveraging an extended measure of Zero-Moment Point (ZMP)-driven rewards and task-driven rewards in a whole-body actor-critic framework. Experiments conducted on a full-sized Unitree H1-2 robot verify the ability of our method to maintain balance on extremely narrow terrains.
  arXiv  Detail & Related papers  (2025-02-24T14:53:45Z)
• SpikingSoft: A Spiking Neuron Controller for Bio-inspired Locomotion with Soft Snake Robots [9.358725923314006]
  This work explores the possibility of generating locomotion gaits by utilizing physical oscillations in a soft snake. We introduce the Double Threshold Spiking neuron model with adjustable thresholds to generate varied output patterns. We demonstrate that our approach, termed SpikingSoft, naturally pairs and integrates with reinforcement learning.
  arXiv  Detail & Related papers  (2025-01-31T12:00:54Z)
• Learning to enhance multi-legged robot on rugged landscapes [7.956679144631909]
  Multi-legged robots offer a promising solution forNavigating rugged landscapes. Recent studies have shown that a linear controller can ensure reliable mobility on challenging terrains. We develop a MuJoCo-based simulator tailored to this robotic platform and use the simulation to develop a reinforcement learning-based control framework.
  arXiv  Detail & Related papers  (2024-09-14T15:53:08Z)
• Evolution and learning in differentiable robots [0.0]
  We use differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans. Non-differentiable changes to the mechanical structure of each robot in the population were applied by a genetic algorithm in an outer loop of search. One of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior.
  arXiv  Detail & Related papers  (2024-05-23T15:45:43Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• Reinforcement learning for freeform robot design [0.0]
  We show policy gradients for designing freeform robots with arbitrary external and internal structure. This is achieved through actions that deposit or remove bundles of atomic building blocks to form higher-level nonparametric macrostructures such as appendages, organs and cavities.
  arXiv  Detail & Related papers  (2023-10-09T12:39:44Z)
• Hierarchical generative modelling for autonomous robots [8.023920215148486]
  We show how a humanoid robot can autonomously complete a complex task that requires a holistic use of locomotion, manipulation, and grasping. Specifically, we demonstrate the ability of a humanoid robot that can retrieve and transport a box, open and walk through a door to reach the destination, approach and kick a football, while showing robust performance in presence of body damage and ground irregularities.
  arXiv  Detail & Related papers  (2023-08-15T13:51:03Z)
• Barkour: Benchmarking Animal-level Agility with Quadruped Robots [70.97471756305463]
  We introduce the Barkour benchmark, an obstacle course to quantify agility for legged robots. Inspired by dog agility competitions, it consists of diverse obstacles and a time based scoring mechanism. We present two methods for tackling the benchmark.
  arXiv  Detail & Related papers  (2023-05-24T02:49:43Z)
• Simultaneous Contact-Rich Grasping and Locomotion via Distributed Optimization Enabling Free-Climbing for Multi-Limbed Robots [60.06216976204385]
  We present an efficient motion planning framework for simultaneously solving locomotion, grasping, and contact problems. We demonstrate our proposed framework in the hardware experiments, showing that the multi-limbed robot is able to realize various motions including free-climbing at a slope angle 45deg with a much shorter planning time.
  arXiv  Detail & Related papers  (2022-07-04T13:52:10Z)
• VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation [78.92147339883137]
  We show that it is pivotal in increasing controller robustness by learning a latent space capturing the key stance phases constituting a particular gait. We demonstrate that specific properties of the drive signal map directly to gait parameters such as cadence, footstep height and full stance duration. The use of a generative model facilitates the detection and mitigation of disturbances to provide a versatile and robust planning framework.
  arXiv  Detail & Related papers  (2022-05-02T19:49:53Z)
• Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion [69.87112582900363]
  Current planners are unable to vary key gait parameters continuously while the robot is in motion. In this work we address this limitation by learning a latent space capturing the key stance phases constituting a particular gait. We demonstrate that specific properties of the drive signal map directly to gait parameters such as cadence, foot step height and full stance duration.
  arXiv  Detail & Related papers  (2021-12-09T10:02:02Z)
• An Adaptable Approach to Learn Realistic Legged Locomotion without Examples [38.81854337592694]
  This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded inverted pendulum model as a reference. We present experimental results showing that even in a model-free setup, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot.
  arXiv  Detail & Related papers  (2021-10-28T10:14:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.