Blocks Assemble! Learning to Assemble with Large-Scale Structured
Reinforcement Learning
- URL: http://arxiv.org/abs/2203.13733v1
- Date: Tue, 15 Mar 2022 18:21:02 GMT
- Title: Blocks Assemble! Learning to Assemble with Large-Scale Structured
Reinforcement Learning
- Authors: Seyed Kamyar Seyed Ghasemipour, Daniel Freeman, Byron David, Shixiang
(Shane) Gu, Satoshi Kataoka, Igor Mordatch
- Abstract summary: Assembly of multi-part physical structures is a valuable end product for autonomous robotics.
We introduce a naturalistic physics-based environment with a set of connectable magnet blocks inspired by children's toy kits.
We find that the combination of large-scale reinforcement learning and graph-based policies is an effective recipe for training agents.
- Score: 23.85678777628229
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Assembly of multi-part physical structures is both a valuable end product for
autonomous robotics, as well as a valuable diagnostic task for open-ended
training of embodied intelligent agents. We introduce a naturalistic
physics-based environment with a set of connectable magnet blocks inspired by
children's toy kits. The objective is to assemble blocks into a succession of
target blueprints. Despite the simplicity of this objective, the compositional
nature of building diverse blueprints from a set of blocks leads to an
explosion of complexity in structures that agents encounter. Furthermore,
assembly stresses agents' multi-step planning, physical reasoning, and bimanual
coordination. We find that the combination of large-scale reinforcement
learning and graph-based policies -- surprisingly without any additional
complexity -- is an effective recipe for training agents that not only
generalize to complex unseen blueprints in a zero-shot manner, but even operate
in a reset-free setting without being trained to do so. Through extensive
experiments, we highlight the importance of large-scale training, structured
representations, contributions of multi-task vs. single-task learning, as well
as the effects of curriculums, and discuss qualitative behaviors of trained
agents.
Related papers
- Twisting Lids Off with Two Hands [88.20584085182857]
We show that policies trained in simulation using deep reinforcement learning can be effectively transferred to the real world.
Our findings serve as compelling evidence that deep reinforcement learning combined with sim-to-real transfer remains a promising approach for addressing manipulation problems of unprecedented complexity.
arXiv Detail & Related papers (2024-03-04T18:59:30Z) - Modular Neural Network Policies for Learning In-Flight Object Catching
with a Robot Hand-Arm System [55.94648383147838]
We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects.
Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, and (iv) a grasping control policy trained to perform soft catching motions.
We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight
arXiv Detail & Related papers (2023-12-21T16:20:12Z) - Structural Concept Learning via Graph Attention for Multi-Level
Rearrangement Planning [2.7195102129095003]
We propose a deep learning approach to perform multi-level object rearrangement planning for scenes with structural dependency hierarchies.
It is trained on a self-generated simulation data set with intuitive structures and works for unseen scenes with an arbitrary number of objects.
We compare our method with a range of classical and model-based baselines to show that our method leverages its scene understanding to achieve better performance, flexibility, and efficiency.
arXiv Detail & Related papers (2023-09-05T19:35:44Z) - Policy Architectures for Compositional Generalization in Control [71.61675703776628]
We introduce a framework for modeling entity-based compositional structure in tasks.
Our policies are flexible and can be trained end-to-end without requiring any action primitives.
arXiv Detail & Related papers (2022-03-10T06:44:24Z) - Brick-by-Brick: Combinatorial Construction with Deep Reinforcement
Learning [52.85981207514049]
We introduce a novel formulation, complex construction, which requires a building agent to assemble unit primitives sequentially.
To construct a target object, we provide incomplete knowledge about the desired target (i.e., 2D images) instead of exact and explicit information to the agent.
We demonstrate that the proposed method successfully learns to construct an unseen object conditioned on a single image or multiple views of a target object.
arXiv Detail & Related papers (2021-10-29T01:09:51Z) - Learning to Design and Construct Bridge without Blueprint [20.524052738716435]
We study a new challenging assembly task, designing and constructing a bridge without a blueprint.
In this task, the robot needs to first design a feasible bridge architecture for arbitrarily wide cliffs and then manipulate the blocks reliably to construct a stable bridge according to the proposed design.
At the high level, the system learns a bridge blueprint policy in a physical simulator using deep reinforcement learning and curriculum learning.
For low-level control, we implement a motion-planning-based policy for real-robot motion control, which can be directly combined with a trained blueprint policy for real-world bridge construction without tuning.
arXiv Detail & Related papers (2021-08-05T08:17:22Z) - A Consciousness-Inspired Planning Agent for Model-Based Reinforcement
Learning [104.3643447579578]
We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state.
The design allows agents to learn to plan effectively, by attending to the relevant objects, leading to better out-of-distribution generalization.
arXiv Detail & Related papers (2021-06-03T19:35:19Z) - CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and
Transfer Learning [138.40338621974954]
CausalWorld is a benchmark for causal structure and transfer learning in a robotic manipulation environment.
Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures.
arXiv Detail & Related papers (2020-10-08T23:01:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.