BrickNet: Graph-Backed Generative Brick Assembly

• URL: http://arxiv.org/abs/2604.22984v1
• Date: Fri, 24 Apr 2026 19:55:51 GMT
• Title: BrickNet: Graph-Backed Generative Brick Assembly
• Abstract summary: We train a language model to generate LEGO-brick build sequences.<n>We collect a large-scale dataset of over 100,000 human-designed LDraw brick objects and scenes.<n>When predicting block pose directly, build sequences quickly become invalid after a small number of steps.
• Score: 54.88688542316808
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We train a language model to generate LEGO-brick build sequences. While prior work has been restricted to discrete, voxel-like towers, we consider a much broader set of pieces, encompassing thousands of part types with diverse connection semantics. To enable this, we first collect a large-scale dataset of over 100,000 human-designed LDraw brick objects and scenes. The complexity of our setting makes it challenging to autoregressively assemble structures that satisfy physical constraints. When predicting block pose directly, build sequences quickly become invalid after a small number of steps. Although pieces are placed in 3D space, it is the spatial relationships of the parts which define the whole. With this in mind, we design a graph-based program representation that parametrizes structure through connectivity, improving the physical grounding of generated sequences. To enable future applications, we make our dataset and models available for research purposes. https://kulits.github.io/BrickNet

Related papers

• MegaParts: Scaling Part-Aware 3D Object Generation to 300 Parts via Token-Efficient Autoregressive Modeling [48.6816265114274]
  MegaParts is a scalable autoregressive 3D generation framework.<n>It scales to objects with up to 300 parts and sequence lengths up to 256k tokens.<n>Our method achieves higher mesh quality than baseline autoregressive and diffusion models.
  arXiv  Detail & Related papers  (2026-08-14T18:00:02Z)
• Generating Physically Stable and Buildable Brick Structures from Text [63.75381708299733]
  BrickGPT is the first approach for generating physically stable assembly models from text prompts.<n>We release our dataset, StableText2Brick, containing over7,000 3D textured brick structures.
  arXiv  Detail & Related papers  (2025-05-08T17:58:18Z)
• TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly [51.29305265324916]
  We propose a class-agnostic tree-transformer framework to predict the sequential assembly actions from input multi-view images. A major challenge of the sequential brick assembly task is that the step-wise action labels are costly and tedious to obtain in practice. We mitigate this problem by leveraging synthetic-to-real transfer learning.
  arXiv  Detail & Related papers  (2024-07-22T14:05:27Z)
• Budget-Aware Sequential Brick Assembly with Efficient Constraint Satisfaction [63.672314717599285]
  We tackle the problem of sequential brick assembly with LEGO bricks to create 3D structures. In particular, the number of assemblable structures increases exponentially as the number of bricks used increases. We propose a new method to predict the scores of the next brick position by employing a U-shaped sparse 3D convolutional neural network.
  arXiv  Detail & Related papers  (2022-10-03T15:35:08Z)
• Break and Make: Interactive Structural Understanding Using LEGO Bricks [61.01136603613139]
  We build a fully interactive 3D simulator that allows learning agents to assemble, disassemble and manipulate LEGO models. We take a first step towards solving this problem using sequence-to-sequence models.
  arXiv  Detail & Related papers  (2022-07-27T18:33:09Z)
• Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning [52.85981207514049]
  We introduce a novel formulation, complex construction, which requires a building agent to assemble unit primitives sequentially. To construct a target object, we provide incomplete knowledge about the desired target (i.e., 2D images) instead of exact and explicit information to the agent. We demonstrate that the proposed method successfully learns to construct an unseen object conditioned on a single image or multiple views of a target object.
  arXiv  Detail & Related papers  (2021-10-29T01:09:51Z)
• BuildingNet: Learning to Label 3D Buildings [19.641000866952815]
  BuildingNet: (a) large-scale 3D building models whose exteriors consistently labeled, (b) a neural network that labels building analyzing and structural relations of their geometric primitives. The dataset covers categories, such as houses, churches, skyscrapers, town halls and castles.
  arXiv  Detail & Related papers  (2021-10-11T01:45:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.