A Lightweight and Transferable Design for Robust LEGO Manipulation
- URL: http://arxiv.org/abs/2309.02354v3
- Date: Fri, 19 Apr 2024 05:18:13 GMT
- Title: A Lightweight and Transferable Design for Robust LEGO Manipulation
- Authors: Ruixuan Liu, Yifan Sun, Changliu Liu,
- Abstract summary: This paper investigates safe and efficient robotic Lego manipulation.
An end-of-arm tool (EOAT) is designed, which reduces the problem dimension and allows large industrial robots to manipulate small Lego bricks.
Experiments demonstrate that the EOAT can reliably manipulate Lego bricks and the learning framework can effectively and safely improve the manipulation performance to a 100% success rate.
- Score: 10.982854061044339
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Lego is a well-known platform for prototyping pixelized objects. However, robotic Lego prototyping (i.e., manipulating Lego bricks) is challenging due to the tight connections and accuracy requirements. This paper investigates safe and efficient robotic Lego manipulation. In particular, this paper reduces the complexity of the manipulation by hardware-software co-design. An end-of-arm tool (EOAT) is designed, which reduces the problem dimension and allows large industrial robots to manipulate small Lego bricks. In addition, this paper uses evolution strategy to optimize the robot motion for Lego manipulation. Experiments demonstrate that the EOAT can reliably manipulate Lego bricks and the learning framework can effectively and safely improve the manipulation performance to a 100% success rate. The co-design is deployed to multiple robots (i.e., FANUC LR-mate 200id/7L and Yaskawa GP4) to demonstrate its generalizability and transferability. In the end, we show that the proposed solution enables sustainable robotic Lego prototyping, in which the robot can repeatedly assemble and disassemble different prototypes.
Related papers
- Learning to Build by Building Your Own Instructions [56.734927320020496]
We develop a new technique for the recently proposed Break-and-Make problem in LTRON.
An agent must learn to build a previously unseen LEGO assembly using a single interactive session.
We train these models using online imitation learning which allows the model to learn from its own mistakes.
arXiv Detail & Related papers (2024-10-01T22:39:58Z) - Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Reset [22.660709957291747]
We introduce a novel problem: Generative Design-for-Robot-Assembly (GDfRA)
The task is to generate an assembly based on a natural language prompt (e.g., ''giraffe'') and an image of available physical components, such as 3D-printed blocks.
We present Blox-Net, a GDfRA system that combines generative vision language models with well-established methods in computer vision, perturbation analysis, motion planning, and physical robot experimentation.
arXiv Detail & Related papers (2024-09-25T17:42:20Z) - BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO [5.052293146674793]
We present BricksRL, a platform designed to democratize access to robotics for reinforcement learning research and education.
BricksRL facilitates the creation, design, and training of custom LEGO robots in the real world by interfacing them with the TorchRL library for reinforcement learning agents.
arXiv Detail & Related papers (2024-06-25T12:17:44Z) - RoboScript: Code Generation for Free-Form Manipulation Tasks across Real
and Simulation [77.41969287400977]
This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation.
We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language.
We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
arXiv Detail & Related papers (2024-02-22T15:12:00Z) - Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling [98.65190562585461]
Diffusion models excel at generating photo-realistic images but come with significant computational costs.
Current options like U-Net and Vision Transformer often rely on resource-intensive deep networks.
This study introduces LEGO bricks, which seamlessly integrate Local-feature Enrichment and Global-content Orchestration.
arXiv Detail & Related papers (2023-10-10T07:52:30Z) - WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Model [92.90127398282209]
This paper investigates the potential of integrating the most recent Large Language Models (LLMs) and existing visual grounding and robotic grasping system.
We introduce the WALL-E (Embodied Robotic WAiter load lifting with Large Language model) as an example of this integration.
We deploy this LLM-empowered system on the physical robot to provide a more user-friendly interface for the instruction-guided grasping task.
arXiv Detail & Related papers (2023-08-30T11:35:21Z) - Break and Make: Interactive Structural Understanding Using LEGO Bricks [61.01136603613139]
We build a fully interactive 3D simulator that allows learning agents to assemble, disassemble and manipulate LEGO models.
We take a first step towards solving this problem using sequence-to-sequence models.
arXiv Detail & Related papers (2022-07-27T18:33:09Z) - Learning Design and Construction with Varying-Sized Materials via
Prioritized Memory Resets [30.993174896902357]
Can a robot autonomously learn to design and construct a bridge from varying-sized blocks without a blueprint?
It is a challenging task with long horizon and sparse reward -- the robot has to figure out physically stable design schemes and feasible actions to manipulate and transport blocks.
In this paper, we propose a hierarchical approach for this problem. It consists of a reinforcement-learning designer to propose high-level building instructions and a motion-planning-based action generator to manipulate blocks at the low level.
arXiv Detail & Related papers (2022-04-12T03:45:48Z) - Image2Lego: Customized LEGO Set Generation from Images [50.87935634904456]
We implement a system that generates a LEGO brick model from 2D images.
Models are obtained by algorithmic conversion of the 3D voxelized model to bricks.
We generate step-by-step building instructions and animations for LEGO models of objects and human faces.
arXiv Detail & Related papers (2021-08-19T03:42:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.