Prompt-to-Product: Generative Assembly via Bimanual Manipulation

• URL: http://arxiv.org/abs/2508.21063v1
• Date: Thu, 28 Aug 2025 17:59:05 GMT
• Title: Prompt-to-Product: Generative Assembly via Bimanual Manipulation
• Authors: Ruixuan Liu, Philip Huang, Ava Pun, Kangle Deng, Shobhit Aggarwal, Kevin Tang, Michelle Liu, Deva Ramanan, Jun-Yan Zhu, Jiaoyang Li, Changliu Liu,
• Abstract summary: Prompt-to-Product is an automated pipeline that generates real-world assembly products from natural language prompts.<n>We leverage LEGO bricks as the assembly platform and automate the process of creating brick assembly structures.
• Score: 67.08306531337634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creating assembly products demands significant manual effort and expert knowledge in 1) designing the assembly and 2) constructing the product. This paper introduces Prompt-to-Product, an automated pipeline that generates real-world assembly products from natural language prompts. Specifically, we leverage LEGO bricks as the assembly platform and automate the process of creating brick assembly structures. Given the user design requirements, Prompt-to-Product generates physically buildable brick designs, and then leverages a bimanual robotic system to construct the real assembly products, bringing user imaginations into the real world. We conduct a comprehensive user study, and the results demonstrate that Prompt-to-Product significantly lowers the barrier and reduces manual effort in creating assembly products from imaginative ideas.

Related papers

• Prompt-to-Parts: Generative AI for Physical Assembly and Scalable Instructions [3.0620527758972496]
  We present a framework for generating physically realizable assembly instructions from natural language descriptions.<n>Using LDraw as a text-rich intermediate representation, we demonstrate that large language models can be guided with tools to produce valid step-by-step construction sequences.<n>We introduce a Python library for programmatic model generation and evaluate buildable outputs on complex satellites, aircraft, and architectural domains.
  arXiv  Detail & Related papers  (2025-12-10T05:55:33Z)
• Agentic Design of Compositional Machines [26.167638081496914]
  We investigate whether large language models (LLMs) can learn to create machines.<n>We use BesiegeField, a testbed built on the machine-building game Besiege.<n>We benchmark state-of-the-art RLs with agentic and identify key capabilities required for success.
  arXiv  Detail & Related papers  (2025-10-16T17:59:58Z)
• No More Manual Guides: Automatic and Scalable Generation of High-Quality Excel Tutorials [63.10037761131196]
  Existing tutorials are manually authored by experts, require frequent updates after each software release, and incur substantial labor costs.<n>We present the first framework for automatically generating Excel tutorials directly from natural language task descriptions.<n>Our framework improves task execution success rates by 8.5% over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-09-26T03:21:39Z)
• Generating Physically Stable and Buildable Brick Structures from Text [63.75381708299733]
  BrickGPT is the first approach for generating physically stable assembly models from text prompts.<n>We release our dataset, StableText2Brick, containing over7,000 3D textured brick structures.
  arXiv  Detail & Related papers  (2025-05-08T17:58:18Z)
• AssemblyComplete: 3D Combinatorial Construction with Deep Reinforcement Learning [4.3507834596906125]
  A critical goal in robotics is to teach robots to adapt to real-world collaborative tasks, particularly in automatic assembly. This paper introduces 3D assembly completion, which is demonstrated using unit primitives (i.e., Lego bricks) We propose a two-part deep reinforcement learning (DRL) framework that tackles teaching the robot to understand the objective of an incomplete assembly and learning a construction policy to complete the assembly.
  arXiv  Detail & Related papers  (2024-10-20T18:51:17Z)
• Learning to Build by Building Your Own Instructions [56.734927320020496]
  We develop a new technique for the recently proposed Break-and-Make problem in LTRON. An agent must learn to build a previously unseen LEGO assembly using a single interactive session. We train these models using online imitation learning which allows the model to learn from its own mistakes.
  arXiv  Detail & Related papers  (2024-10-01T22:39:58Z)
• Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Reset [22.660709957291747]
  We introduce a novel problem: Generative Design-for-Robot-Assembly (GDfRA) The task is to generate an assembly based on a natural language prompt (e.g., ''giraffe'') and an image of available physical components, such as 3D-printed blocks. We present Blox-Net, a GDfRA system that combines generative vision language models with well-established methods in computer vision, perturbation analysis, motion planning, and physical robot experimentation.
  arXiv  Detail & Related papers  (2024-09-25T17:42:20Z)
• Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking [6.919208054874144]
  Combinatorial assembly uses standardized unit primitives to build objects that satisfy user specifications.<n>This paper studies assembly sequence planning (ASP) for physical assembly.<n>We employ deep reinforcement learning to learn a construction policy for placing unit primitives sequentially to build the desired object.
  arXiv  Detail & Related papers  (2024-08-19T17:16:35Z)
• BrickPal: Augmented Reality-based Assembly Instructions for Brick Models [20.585631669565295]
  BrickPal is an augmented reality-based system, which visualizes assembly instructions in an augmented reality head-mounted display. It utilizes Natural Language Processing (NLP) techniques to generate plausible assembly sequences, and provide real-time guidance in the AR headset. Our user study demonstrates BrickPal's effectiveness at assisting users in brick assembly compared to traditional assembly methods.
  arXiv  Detail & Related papers  (2023-07-06T17:42:56Z)
• Break and Make: Interactive Structural Understanding Using LEGO Bricks [61.01136603613139]
  We build a fully interactive 3D simulator that allows learning agents to assemble, disassemble and manipulate LEGO models. We take a first step towards solving this problem using sequence-to-sequence models.
  arXiv  Detail & Related papers  (2022-07-27T18:33:09Z)
• Image2Lego: Customized LEGO Set Generation from Images [50.87935634904456]
  We implement a system that generates a LEGO brick model from 2D images. Models are obtained by algorithmic conversion of the 3D voxelized model to bricks. We generate step-by-step building instructions and animations for LEGO models of objects and human faces.
  arXiv  Detail & Related papers  (2021-08-19T03:42:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.