Sample-Efficient Co-Design of Robotic Agents Using Multi-fidelity Training on Universal Policy Network

• URL: http://arxiv.org/abs/2309.04085v1
• Date: Fri, 8 Sep 2023 02:54:31 GMT
• Title: Sample-Efficient Co-Design of Robotic Agents Using Multi-fidelity Training on Universal Policy Network
• Authors: Kishan R. Nagiredla, Buddhika L. Semage, Thommen G. Karimpanal, Arun Kumar A. V and Santu Rana
• Abstract summary: We propose a multi-fidelity-based design exploration strategy based on Hyperband. We tie the controllers learnt across the design spaces through a universal learner policy for warm-starting the subsequent controller learning problems. Experiments performed on a wide range of agent design problems demonstrate the superiority of our method compared to the baselines.
• Score: 12.283890343327233
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Co-design involves simultaneously optimizing the controller and agents physical design. Its inherent bi-level optimization formulation necessitates an outer loop design optimization driven by an inner loop control optimization. This can be challenging when the design space is large and each design evaluation involves data-intensive reinforcement learning process for control optimization. To improve the sample-efficiency we propose a multi-fidelity-based design exploration strategy based on Hyperband where we tie the controllers learnt across the design spaces through a universal policy learner for warm-starting the subsequent controller learning problems. Further, we recommend a particular way of traversing the Hyperband generated design matrix that ensures that the stochasticity of the Hyperband is reduced the most with the increasing warm starting effect of the universal policy learner as it is strengthened with each new design evaluation. Experiments performed on a wide range of agent design problems demonstrate the superiority of our method compared to the baselines. Additionally, analysis of the optimized designs shows interesting design alterations including design simplifications and non-intuitive alterations that have emerged in the biological world.

Related papers

• ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models [5.642568057913696]
  This paper presents ADO-LLM, the first work integrating large language models (LLMs) with Bayesian Optimization for analog design optimization. ADO-LLM leverages the LLM's ability to infuse domain knowledge to rapidly generate viable design points to remedy BO's inefficiency in finding high value design areas. We evaluate the proposed framework on two different types of analog circuits and demonstrate notable improvements in design efficiency and effectiveness.
  arXiv  Detail & Related papers  (2024-06-26T21:42:50Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design [7.066023612489374]
  We propose a new collaborative paradigm for Bayesian optimization. Our approach provides a generic and flexible framework that can incorporate different collaboration mechanisms. We show that our framework can effectively accelerate and improve the optimal design process and benefit all participants.
  arXiv  Detail & Related papers  (2023-06-25T21:43:05Z)
• Meta Reinforcement Learning for Optimal Design of Legged Robots [9.054187238463212]
  We present a design optimization framework using model-free meta reinforcement learning. We show that our approach allows higher performance while not being constrained by predefined motions or gait patterns.
  arXiv  Detail & Related papers  (2022-10-06T08:37:52Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• Machine Learning Framework for Quantum Sampling of Highly-Constrained, Continuous Optimization Problems [101.18253437732933]
  We develop a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate unconstrained binary optimization problems. We showcase the framework's performance on two inverse design problems by optimizing thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering.
  arXiv  Detail & Related papers  (2021-05-06T02:22:23Z)
• Reinforcement Learning for Flexibility Design Problems [77.37213643948108]
  We develop a reinforcement learning framework for flexibility design problems. Empirical results show that the RL-based method consistently finds better solutions than classical methods.
  arXiv  Detail & Related papers  (2021-01-02T02:44:39Z)
• Optimization-Inspired Learning with Architecture Augmentations and Control Mechanisms for Low-Level Vision [74.9260745577362]
  This paper proposes a unified optimization-inspired learning framework to aggregate Generative, Discriminative, and Corrective (GDC) principles. We construct three propagative modules to effectively solve the optimization models with flexible combinations. Experiments across varied low-level vision tasks validate the efficacy and adaptability of GDC.
  arXiv  Detail & Related papers  (2020-12-10T03:24:53Z)
• Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs [5.8010446129208155]
  This study proposes a reinforcement learning based generative design process, with reward functions maximizing the diversity of topology designs. We show that RL-based generative design produces a large number of diverse designs within a short inference time by exploiting GPU in a fully automated manner.
  arXiv  Detail & Related papers  (2020-08-17T06:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.