RoboMorph: Evolving Robot Morphology using Large Language Models

• URL: http://arxiv.org/abs/2407.08626v1
• Date: Thu, 11 Jul 2024 16:05:56 GMT
• Title: RoboMorph: Evolving Robot Morphology using Large Language Models
• Authors: Kevin Qiu, Krzysztof Ciebiera, Paweł Fijałkowski, Marek Cygan, Łukasz Kuciński,
• Abstract summary: We introduce RoboMorph, an automated approach for generating and optimizing modular robot designs. By integrating automatic prompt design and a reinforcement learning based control algorithm, RoboMorph iteratively improves robot designs through feedback loops.
• Score: 0.5812095716568273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce RoboMorph, an automated approach for generating and optimizing modular robot designs using large language models (LLMs) and evolutionary algorithms. In this framework, we represent each robot design as a grammar and leverage the capabilities of LLMs to navigate the extensive robot design space, which is traditionally time-consuming and computationally demanding. By integrating automatic prompt design and a reinforcement learning based control algorithm, RoboMorph iteratively improves robot designs through feedback loops. Our experimental results demonstrate that RoboMorph can successfully generate nontrivial robots that are optimized for a single terrain while showcasing improvements in morphology over successive evolutions. Our approach demonstrates the potential of using LLMs for data-driven and modular robot design, providing a promising methodology that can be extended to other domains with similar design frameworks.

Related papers

• On the Exploration of LM-Based Soft Modular Robot Design [26.847859137653487]
  Large language models (LLMs) have demonstrated promising capabilities in modeling real-world knowledge. In this paper, we explore the potential of using LLMs to aid in the design of soft modular robots. Our model performs well in evaluations for designing soft modular robots with uni- and bi-directional and stair-descending capabilities.
  arXiv  Detail & Related papers  (2024-11-01T04:03:05Z)
• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Model [72.66465487508556]
  DiffGen is a novel framework that integrates differentiable physics simulation, differentiable rendering, and a vision-language model. It can generate realistic robot demonstrations by minimizing the distance between the embedding of the language instruction and the embedding of the simulated observation. Experiments demonstrate that with DiffGen, we could efficiently and effectively generate robot data with minimal human effort or training time.
  arXiv  Detail & Related papers  (2024-05-12T15:38:17Z)
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
  This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
  arXiv  Detail & Related papers  (2024-02-22T15:12:00Z)
• DittoGym: Learning to Control Soft Shape-Shifting Robots [30.287452037945542]
  We explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime. We formalize control of reconfigurable soft robots as a high-dimensional reinforcement learning (RL) problem. We introduce DittoGym, a comprehensive RL benchmark for reconfigurable soft robots that require fine-grained morphology changes.
  arXiv  Detail & Related papers  (2024-01-24T05:03:05Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation [68.70755196744533]
  RoboGen is a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics.
  arXiv  Detail & Related papers  (2023-11-02T17:59:21Z)
• Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design [40.01142267374432]
  Multi-cellular robot design aims to create robots comprised of numerous cells that can be efficiently controlled to perform diverse tasks. Previous research has demonstrated the ability to generate robots for various tasks, but these approaches often optimize robots directly in the vast design space. This paper presents a novel coarse-to-fine method for designing multi-cellular robots.
  arXiv  Detail & Related papers  (2023-11-01T11:56:32Z)
• GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation [16.96128900256427]
  We present Grammar-guided Latent Space Optimization (GLSO), a framework that transforms design automation into a low-dimensional continuous optimization problem. In this work, we present a framework that transforms design automation into a low-dimensional continuous optimization problem by training a graph variational autoencoder (VAE) to learn a mapping between the graph-structured design space and a continuous latent space.
  arXiv  Detail & Related papers  (2022-09-23T17:48:24Z)
• MetaMorph: Learning Universal Controllers with Transformers [45.478223199658785]
  In robotics we primarily train a single robot for a single task. modular robot systems now allow for the flexible combination of general-purpose building blocks into task optimized morphologies. We propose MetaMorph, a Transformer based approach to learn a universal controller over a modular robot design space.
  arXiv  Detail & Related papers  (2022-03-22T17:58:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.