Reinforcement learning for freeform robot design

• URL: http://arxiv.org/abs/2310.05670v2
• Date: Fri, 1 Mar 2024 19:20:18 GMT
• Title: Reinforcement learning for freeform robot design
• Abstract summary: We show policy gradients for designing freeform robots with arbitrary external and internal structure. This is achieved through actions that deposit or remove bundles of atomic building blocks to form higher-level nonparametric macrostructures such as appendages, organs and cavities.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inspired by the necessity of morphological adaptation in animals, a growing body of work has attempted to expand robot training to encompass physical aspects of a robot's design. However, reinforcement learning methods capable of optimizing the 3D morphology of a robot have been restricted to reorienting or resizing the limbs of a predetermined and static topological genus. Here we show policy gradients for designing freeform robots with arbitrary external and internal structure. This is achieved through actions that deposit or remove bundles of atomic building blocks to form higher-level nonparametric macrostructures such as appendages, organs and cavities. Although results are provided for open loop control only, we discuss how this method could be adapted for closed loop control and sim2real transfer to physical machines in future.

Related papers

• Reinforcement learning in linear embedding space unlocks generalizable control across soft robot configurations [60.42735420750131]
  Soft-bodied organisms such as octopuses and elephant trunks exhibit remarkable morphological adaptability.<n>Various soft robots have emerged in recent decades, featuring diverse materials, stiffnesses, and morphologies tailored to specific tasks.<n>Here, we introduce a generalizable control system that enables rapid adaptation across diverse soft robot configurations.
  arXiv  Detail & Related papers  (2026-06-06T11:09:46Z)
• Creating manufacturable blueprints for coarse-grained virtual robots [7.863042150602543]
  We automate the mapping of robot design spaces into blueprints that can be followed by a builder.<n>The pipeline incrementally resolves manufacturing constraints by embedding the structural and functional semantics of motors, electronics, batteries, and wiring into the abstract virtual design.<n>In lieu of evolution, a user-defined or AI-generated sketch'' of a body plan can also be fed as input to the pipeline, providing a versatile framework for accelerating the design of novel robots.
  arXiv  Detail & Related papers  (2026-03-13T20:44:54Z)
• Shape-Interpretable Visual Self-Modeling Enables Geometry-Aware Continuum Robot Control [10.253290204273094]
  Continuum robots possess high flexibility and redundancy, making them well suited for safe interaction in complex environments.<n>Existing vision-based control approaches often rely on end-to-end learning, achieving shape regulation without explicit awareness of robot geometry.<n>Here, we introduce a shape-interpretable visual self-modeling framework for continuum robots that enables geometry-aware control.
  arXiv  Detail & Related papers  (2026-03-02T11:20:28Z)
• Scalable and General Whole-Body Control for Cross-Humanoid Locomotion [47.617416398266506]
  We introduce XHugWBC, a novel cross-embodiment training framework for humanoid robots.<n>XHugWBC internalizes a broad distribution of morphological and dynamical characteristics during training.<n> Experiments on twelve simulated humanoids and seven real-world robots demonstrate the strong generalization and robustness of the resulting universal controller.
  arXiv  Detail & Related papers  (2026-02-05T15:48:15Z)
• Aggregating swarms through morphology handling design contingencies: from the sweet spot to a rich expressivity [0.12744523252873352]
  We build on a faithful physical model of the self-aligning dynamics of the robots.<n>We show that exploring a range of self-alignment strength allows for a rich expressivity of collective behaviors.
  arXiv  Detail & Related papers  (2026-01-12T14:58:47Z)
• RobotSeg: A Model and Dataset for Segmenting Robots in Image and Video [56.9581053843815]
  We introduce RobotSeg, a foundation model for robot segmentation in image and video.<n>It addresses the lack of adaptation to articulated robots, reliance on manual prompts, and the need for per-frame training mask annotations.<n>It achieves state-of-the-art performance on both images and videos.
  arXiv  Detail & Related papers  (2025-11-28T07:51:02Z)
• Toward Humanoid Brain-Body Co-design: Joint Optimization of Control and Morphology for Fall Recovery [55.951691393378354]
  We propose RoboCraft, a scalable humanoid co-design framework for fall recovery.<n>A shared policy pretrained across multiple designs is progressively finetuned on high-performing morphologies.<n>Experiments show that RoboCraft achieves an average performance gain of 44.55% on seven public humanoid robots.
  arXiv  Detail & Related papers  (2025-10-25T15:40:18Z)
• Body Transformer: Leveraging Robot Embodiment for Policy Learning [51.531793239586165]
  Body Transformer (BoT) is an architecture that leverages the robot embodiment by providing an inductive bias that guides the learning process. We represent the robot body as a graph of sensors and actuators, and rely on masked attention to pool information throughout the architecture. The resulting architecture outperforms the vanilla transformer, as well as the classical multilayer perceptron, in terms of task completion, scaling properties, and computational efficiency.
  arXiv  Detail & Related papers  (2024-08-12T17:31:28Z)
• Controlling diverse robots by inferring Jacobian fields with deep networks [48.279199537720714]
  Mirroring the complex structures and diverse functions of natural organisms is a long-standing challenge in robotics.<n>We introduce a method that uses deep neural networks to map a video stream of a robot to its visuomotor Jacobian field.<n>Our approach achieves accurate closed-loop control and recovers the causal dynamic structure of each robot.
  arXiv  Detail & Related papers  (2024-07-11T17:55:49Z)
• Evolution and learning in differentiable robots [0.0]
  We use differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans. Non-differentiable changes to the mechanical structure of each robot in the population were applied by a genetic algorithm in an outer loop of search. One of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior.
  arXiv  Detail & Related papers  (2024-05-23T15:45:43Z)
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
  This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
  arXiv  Detail & Related papers  (2024-02-22T15:12:00Z)
• DittoGym: Learning to Control Soft Shape-Shifting Robots [30.287452037945542]
  We explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime. We formalize control of reconfigurable soft robots as a high-dimensional reinforcement learning (RL) problem. We introduce DittoGym, a comprehensive RL benchmark for reconfigurable soft robots that require fine-grained morphology changes.
  arXiv  Detail & Related papers  (2024-01-24T05:03:05Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design [40.01142267374432]
  Multi-cellular robot design aims to create robots comprised of numerous cells that can be efficiently controlled to perform diverse tasks. Previous research has demonstrated the ability to generate robots for various tasks, but these approaches often optimize robots directly in the vast design space. This paper presents a novel coarse-to-fine method for designing multi-cellular robots.
  arXiv  Detail & Related papers  (2023-11-01T11:56:32Z)
• Universal Morphology Control via Contextual Modulation [52.742056836818136]
  Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control. Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies. We propose a hierarchical architecture to better model this dependency via contextual modulation.
  arXiv  Detail & Related papers  (2023-02-22T00:04:12Z)
• REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer [57.045140028275036]
  We consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology. Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots. We propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator.
  arXiv  Detail & Related papers  (2022-02-10T18:50:25Z)
• Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms [60.59764170868101]
  Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
  arXiv  Detail & Related papers  (2021-03-05T14:16:20Z)
• Diversity-based Design Assist for Large Legged Robots [4.505477982701834]
  We explore the design space of a class of large legged robots, which stand at around 2m tall and whose design and construction is not well-studied. A novel robot encoding allows for bio-inspired features such as legs scaling along the length of the body.
  arXiv  Detail & Related papers  (2020-04-17T03:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.