Meta Reinforcement Learning for Optimal Design of Legged Robots

• URL: http://arxiv.org/abs/2210.02750v1
• Date: Thu, 6 Oct 2022 08:37:52 GMT
• Title: Meta Reinforcement Learning for Optimal Design of Legged Robots
• Authors: \'Alvaro Belmonte-Baeza, Joonho Lee, Giorgio Valsecchi, Marco Hutter
• Abstract summary: We present a design optimization framework using model-free meta reinforcement learning. We show that our approach allows higher performance while not being constrained by predefined motions or gait patterns.
• Score: 9.054187238463212
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The process of robot design is a complex task and the majority of design decisions are still based on human intuition or tedious manual tuning. A more informed way of facing this task is computational design methods where design parameters are concurrently optimized with corresponding controllers. Existing approaches, however, are strongly influenced by predefined control rules or motion templates and cannot provide end-to-end solutions. In this paper, we present a design optimization framework using model-free meta reinforcement learning, and its application to the optimizing kinematics and actuator parameters of quadrupedal robots. We use meta reinforcement learning to train a locomotion policy that can quickly adapt to different designs. This policy is used to evaluate each design instance during the design optimization. We demonstrate that the policy can control robots of different designs to track random velocity commands over various rough terrains. With controlled experiments, we show that the meta policy achieves close-to-optimal performance for each design instance after adaptation. Lastly, we compare our results against a model-based baseline and show that our approach allows higher performance while not being constrained by predefined motions or gait patterns.

Related papers

• Co-Optimization of Robot Design and Control: Enhancing Performance and Understanding Design Complexity [0.8999666725996974]
  Co-optimization of design and control of robots produces a design and control that are both adapted to the task. We show that retraining the controller of a robot with additional resources after the co-optimization process terminates significantly improves the robot's performance.
  arXiv  Detail & Related papers  (2024-09-13T08:18:01Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• ECoDe: A Sample-Efficient Method for Co-Design of Robotic Agents [11.449817465618658]
  Co-designing autonomous robotic agents involves simultaneously optimizing the controller and physical design of the agent. This can be challenging when the design space is large. We propose a multi-fidelity-based exploration strategy to improve the sample efficiency of co-design.
  arXiv  Detail & Related papers  (2023-09-08T02:54:31Z)
• Parameter-Efficient Fine-Tuning Design Spaces [63.954953653386106]
  We present a parameter-efficient fine-tuning design paradigm and discover design patterns that are applicable to different experimental settings. We show experimentally that these methods consistently and significantly outperform investigated parameter-efficient fine-tuning strategies across different backbone models and different tasks in natural language processing.
  arXiv  Detail & Related papers  (2023-01-04T21:00:18Z)
• Design Space Exploration and Explanation via Conditional Variational Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
  This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE) The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
  arXiv  Detail & Related papers  (2022-11-29T17:28:31Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• Physical Design using Differentiable Learned Simulators [9.380022457753938]
  In inverse design, learned forward simulators are combined with gradient-based design optimization. This framework produces high-quality designs by propagating through trajectories of hundreds of steps. Our results suggest that despite some remaining challenges, machine learning-based simulators are maturing to the point where they can support general-purpose design optimization.
  arXiv  Detail & Related papers  (2022-02-01T19:56:39Z)
• Policy Search for Model Predictive Control with Application to Agile Drone Flight [56.24908013905407]
  We propose a policy-search-for-model-predictive-control framework for MPC. Specifically, we formulate the MPC as a parameterized controller, where the hard-to-optimize decision variables are represented as high-level policies. Experiments show that our controller achieves robust and real-time control performance in both simulation and the real world.
  arXiv  Detail & Related papers  (2021-12-07T17:39:24Z)
• An End-to-End Differentiable Framework for Contact-Aware Robot Design [37.715596272425316]
  We build an end-to-end differentiable framework for contact-aware robot design. A novel deformation-based parameterization allows for the design of articulated rigid robots with arbitrary, complex geometry. A differentiable rigid body simulator can handle contact-rich scenarios and computes analytical gradients for a full spectrum of kinematic and dynamic parameters.
  arXiv  Detail & Related papers  (2021-07-15T17:53:44Z)
• Conservative Objective Models for Effective Offline Model-Based Optimization [78.19085445065845]
  Computational design problems arise in a number of settings, from synthetic biology to computer architectures. We propose a method that learns a model of the objective function that lower bounds the actual value of the ground-truth objective on out-of-distribution inputs. COMs are simple to implement and outperform a number of existing methods on a wide range of MBO problems.
  arXiv  Detail & Related papers  (2021-07-14T17:55:28Z)
• Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments [3.5309638744466167]
  This work developed a meta-learning approach that adapts the control policy on the fly to different changing conditions for robust locomotion. The proposed method constantly updates the interaction model, samples feasible sequences of actions of estimated the state-action trajectories, and then applies the optimal actions to maximize the reward.
  arXiv  Detail & Related papers  (2021-01-19T12:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.