Rearrangement Planning for General Part Assembly

• URL: http://arxiv.org/abs/2307.00206v2
• Date: Sun, 3 Sep 2023 00:20:06 GMT
• Title: Rearrangement Planning for General Part Assembly
• Abstract summary: We present General Part Assembly Transformer (GPAT), a transformer-based model architecture that accurately predicts part poses. Our experiments on both 3D CAD models and real-world scans demonstrate GPAT's generalization abilities to novel and diverse target and part shapes.
• Score: 41.0167725449516
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Most successes in autonomous robotic assembly have been restricted to single target or category. We propose to investigate general part assembly, the task of creating novel target assemblies with unseen part shapes. As a fundamental step to a general part assembly system, we tackle the task of determining the precise poses of the parts in the target assembly, which we we term ``rearrangement planning''. We present General Part Assembly Transformer (GPAT), a transformer-based model architecture that accurately predicts part poses by inferring how each part shape corresponds to the target shape. Our experiments on both 3D CAD models and real-world scans demonstrate GPAT's generalization abilities to novel and diverse target and part shapes.

Related papers

• Backward Layout Search for Sequence-Constrained Robotic Assembly [16.81798013856547]
  We propose Backward Layout Search (BLS) for robotic assembly layout planning.<n>BLS assigns initial part poses in reverse assembly order.<n>It produces collision-free executable layouts and reduces step evaluations and search time compared with a matched forward search.
  arXiv  Detail & Related papers  (2026-08-19T02:29:06Z)
• World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models [54.68021681438324]
  We propose World Action Planner, a robot planning system that leverages the reasoning capabilities of Vision-Language Models (VLMs)<n>Our system enables an agent to propose initial action plans and iteratively refine them via optimization and search, reasoning over imagined world model rollouts.<n>We demonstrate that our approach achieves superior performance across compositional tasks, new layouts, and zero-shot generalization scenarios.
  arXiv  Detail & Related papers  (2026-07-30T02:41:52Z)
• AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects [36.01972955466693]
  Existing datasets focus on simplified scenarios, overlooking shape complexities and assembly trajectories in industrial assemblies.<n>We introduce AssemblyBench, a synthetic dataset of 2,789 industrial objects with multimodal instruction manuals, corresponding 3D part models, and part assembly trajectories.<n>We also propose a transformer-based model, AssemblyDyno, which uses the instructional manual and the 3D shape of each part to jointly predict assembly order and part assembly trajectories.
  arXiv  Detail & Related papers  (2026-05-13T00:44:09Z)
• A3D: Adaptive Affordance Assembly with Dual-Arm Manipulation [25.57693108471214]
  We propose A3D, a framework which learns adaptive affordances to identify optimal support and stabilization locations on furniture parts.<n>To handle evolving assembly states, we introduce an adaptive module that uses interaction feedback to adjust dynamically support strategies.<n>Experiments demonstrate that our framework generalizes effectively to diverse part geometries and furniture categories in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-01-16T08:21:42Z)
• Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion [39.08891847512135]
  We present Assembler, a scalable and generalizable framework for 3D part assembly.<n>It handles diverse, in-the-wild objects with varying part counts, geometries, and structures.<n>It achieves state-of-the-art performance on PartNet and is the first to demonstrate high-quality assembly for complex, real-world objects.
  arXiv  Detail & Related papers  (2025-06-20T15:25:20Z)
• Gondola: Grounded Vision Language Planning for Generalizable Robotic Manipulation [62.711546725154314]
  We introduce Gondola, a grounded vision-language planning model based on large language models (LLMs) for generalizable robotic manipulation.<n>G Gondola takes multi-view images and history plans to produce the next action plan with interleaved texts and segmentation masks of target objects and locations.<n>G Gondola outperforms the state-of-the-art LLM-based method across all four levels of the GemBench dataset.
  arXiv  Detail & Related papers  (2025-06-12T20:04:31Z)
• Manual-PA: Learning 3D Part Assembly from Instruction Diagrams [54.555154845137906]
  We present Manual-PA, a transformer-based instruction Manual-guided 3D Part Assembly framework. Our results show that using the diagrams and the order of the parts lead to significant improvements in assembly performance against the state of the art.
  arXiv  Detail & Related papers  (2024-11-27T03:10:29Z)
• Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection [74.40109927350856]
  We present the Part Assembly Sequence Transformer (PAST) to infer assembly sequences from a target blueprint. We then use a motion planner and optimization to generate part movements and contacts. Experimental results show that our approach generalizes better than prior methods.
  arXiv  Detail & Related papers  (2023-12-17T00:47:13Z)
• SAGE: Bridging Semantic and Actionable Parts for GEneralizable Manipulation of Articulated Objects [9.500480417077272]
  We propose a novel framework that bridges semantic and actionable parts of articulated objects to achieve generalizable manipulation under natural language instructions. A part-grounding module maps the semantic parts into so-called Generalizable Actionable Parts (GAParts), which inherently carry information about part motion. An interactive feedback module is incorporated to respond to failures, which closes the loop and increases the robustness of the overall framework.
  arXiv  Detail & Related papers  (2023-12-03T07:22:42Z)
• Part-aware Prototypical Graph Network for One-shot Skeleton-based Action Recognition [57.86960990337986]
  One-shot skeleton-based action recognition poses unique challenges in learning transferable representation from base classes to novel classes. We propose a part-aware prototypical representation for one-shot skeleton-based action recognition. We demonstrate the effectiveness of our method on two public skeleton-based action recognition datasets.
  arXiv  Detail & Related papers  (2022-08-19T04:54:56Z)
• 3D Part Assembly Generation with Instance Encoded Transformer [22.330218525999857]
  We propose a multi-layer transformer-based framework that involves geometric and relational reasoning between parts to update the part poses iteratively. We extend our framework to a new task called in-process part assembly. Our method achieves far more than 10% improvements over the current state-of-the-art in multiple metrics on the public PartNet dataset.
  arXiv  Detail & Related papers  (2022-07-05T02:40:57Z)
• AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation [58.78094823473567]
  AutoGPart builds a supervision space with geometric prior knowledge encoded, and lets the machine to search for the optimal supervisions for a specific segmentation task automatically. We demonstrate that the performance of segmentation networks using simple backbones can be significantly improved when trained with supervisions searched by our method.
  arXiv  Detail & Related papers  (2022-03-13T03:45:58Z)
• 3D Compositional Zero-shot Learning with DeCompositional Consensus [102.7571947144639]
  We argue that part knowledge should be composable beyond the observed object classes. We present 3D Compositional Zero-shot Learning as a problem of part generalization from seen to unseen object classes.
  arXiv  Detail & Related papers  (2021-11-29T16:34:53Z)
• Unsupervised Pose-Aware Part Decomposition for 3D Articulated Objects [68.73163598790255]
  We propose PPD (unsupervised Pose-aware Part Decomposition) to address a novel setting that explicitly targets man-made articulated objects with mechanical joints. We show that category-common prior learning for both part shapes and poses facilitates the unsupervised learning of (1) part decomposition with non-primitive-based implicit representation, and (2) part pose as joint parameters under single-frame shape supervision.
  arXiv  Detail & Related papers  (2021-10-08T23:53:56Z)
• Generative 3D Part Assembly via Dynamic Graph Learning [34.108515032411695]
  Part assembly is a challenging yet crucial task in 3D computer vision and robotics. We propose an assembly-oriented dynamic graph learning framework that leverages an iterative graph neural network as a backbone.
  arXiv  Detail & Related papers  (2020-06-14T04:26:42Z)
• Learning 3D Part Assembly from a Single Image [20.175502864488493]
  We introduce a novel problem, single-image-guided 3D part assembly, along with a learningbased solution. We study this problem in the setting of furniture assembly from a given complete set of parts and a single image depicting the entire assembled object.
  arXiv  Detail & Related papers  (2020-03-21T21:19:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.