Sequential Brick Assembly with Efficient Constraint Satisfaction
- URL: http://arxiv.org/abs/2210.01021v1
- Date: Mon, 3 Oct 2022 15:35:08 GMT
- Title: Sequential Brick Assembly with Efficient Constraint Satisfaction
- Authors: Seokjun Ahn, Jungtaek Kim, Minsu Cho, Jaesik Park
- Abstract summary: We address the problem of generating a sequence of LEGO brick assembly with high-fidelity structures.
Our method performs a brick structure assessment to predict the next brick position and its confidence by employing a U-shaped sparse 3D convolutional network.
Instead of using handcrafted brick assembly datasets, our model is trained with a large number of 3D objects that allow to create a new high-fidelity structure.
- Score: 39.869693447362145
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We address the problem of generating a sequence of LEGO brick assembly with
high-fidelity structures, satisfying physical constraints between bricks. The
assembly problem is challenging since the number of possible structures
increases exponentially with the number of available bricks, complicating the
physical constraints to satisfy across bricks. To tackle this problem, our
method performs a brick structure assessment to predict the next brick position
and its confidence by employing a U-shaped sparse 3D convolutional network. The
convolution filter efficiently validates physical constraints in a
parallelizable and scalable manner, allowing to process of different brick
types. To generate a novel structure, we devise a sampling strategy to
determine the next brick position by considering attachable positions under
physical constraints. Instead of using handcrafted brick assembly datasets, our
model is trained with a large number of 3D objects that allow to create a new
high-fidelity structure. We demonstrate that our method successfully generates
diverse brick structures while handling two different brick types and
outperforms existing methods based on Bayesian optimization, graph generative
model, and reinforcement learning, all of which are limited to a single brick
type.
Related papers
- TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly [51.29305265324916]
We propose a class-agnostic tree-transformer framework to predict the sequential assembly actions from input multi-view images.
A major challenge of the sequential brick assembly task is that the step-wise action labels are costly and tedious to obtain in practice.
We mitigate this problem by leveraging synthetic-to-real transfer learning.
arXiv Detail & Related papers (2024-07-22T14:05:27Z) - Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling [98.65190562585461]
Diffusion models excel at generating photo-realistic images but come with significant computational costs.
Current options like U-Net and Vision Transformer often rely on resource-intensive deep networks.
This study introduces LEGO bricks, which seamlessly integrate Local-feature Enrichment and Global-content Orchestration.
arXiv Detail & Related papers (2023-10-10T07:52:30Z) - Constructive plaquette compilation for the parity architecture [0.4499833362998487]
We present the first constructive compilation algorithm for the parity architecture using plaquettes.
The algorithm builds a rectangular layout of plaquettes, where in each layer of the rectangle at least one constraint is added.
We show how to pick a valid set of constraints and how this decomposition works.
arXiv Detail & Related papers (2023-07-20T06:49:05Z) - BrickPal: Augmented Reality-based Assembly Instructions for Brick Models [20.585631669565295]
BrickPal is an augmented reality-based system, which visualizes assembly instructions in an augmented reality head-mounted display.
It utilizes Natural Language Processing (NLP) techniques to generate plausible assembly sequences, and provide real-time guidance in the AR headset.
Our user study demonstrates BrickPal's effectiveness at assisting users in brick assembly compared to traditional assembly methods.
arXiv Detail & Related papers (2023-07-06T17:42:56Z) - Break and Make: Interactive Structural Understanding Using LEGO Bricks [61.01136603613139]
We build a fully interactive 3D simulator that allows learning agents to assemble, disassemble and manipulate LEGO models.
We take a first step towards solving this problem using sequence-to-sequence models.
arXiv Detail & Related papers (2022-07-27T18:33:09Z) - Brick-by-Brick: Combinatorial Construction with Deep Reinforcement
Learning [52.85981207514049]
We introduce a novel formulation, complex construction, which requires a building agent to assemble unit primitives sequentially.
To construct a target object, we provide incomplete knowledge about the desired target (i.e., 2D images) instead of exact and explicit information to the agent.
We demonstrate that the proposed method successfully learns to construct an unseen object conditioned on a single image or multiple views of a target object.
arXiv Detail & Related papers (2021-10-29T01:09:51Z) - An Integer Linear Programming Framework for Mining Constraints from Data [81.60135973848125]
We present a general framework for mining constraints from data.
In particular, we consider the inference in structured output prediction as an integer linear programming (ILP) problem.
We show that our approach can learn to solve 9x9 Sudoku puzzles and minimal spanning tree problems from examples without providing the underlying rules.
arXiv Detail & Related papers (2020-06-18T20:09:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.