BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly

• URL: http://arxiv.org/abs/2506.06221v2
• Date: Tue, 10 Jun 2025 16:32:34 GMT
• Title: BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly
• Authors: Yan Shen, Ruihai Wu, Yubin Ke, Xinyuan Song, Zeyi Li, Xiaoqi Li, Hongwei Fan, Haoran Lu, Hao dong,
• Abstract summary: Shape assembly, the process of combining parts into a complete whole, is a crucial robotic skill with broad real-world applications.<n>This requires the robot to recognize geometric cues for grasping, assembly, and subsequent bimanual collaborative manipulation on varied fragments.<n>In this paper, we exploit the geometric generalization of point-level affordance, learning affordance aware of bimanual collaboration in geometric assembly with long-horizon action sequences.
• Score: 7.667456216197558
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Shape assembly, the process of combining parts into a complete whole, is a crucial robotic skill with broad real-world applications. Among various assembly tasks, geometric assembly--where broken parts are reassembled into their original form (e.g., reconstructing a shattered bowl)--is particularly challenging. This requires the robot to recognize geometric cues for grasping, assembly, and subsequent bimanual collaborative manipulation on varied fragments. In this paper, we exploit the geometric generalization of point-level affordance, learning affordance aware of bimanual collaboration in geometric assembly with long-horizon action sequences. To address the evaluation ambiguity caused by geometry diversity of broken parts, we introduce a real-world benchmark featuring geometric variety and global reproducibility. Extensive experiments demonstrate the superiority of our approach over both previous affordance-based and imitation-based methods. Project page: https://sites.google.com/view/biassembly/.

Related papers

• Detection Based Part-level Articulated Object Reconstruction from Single RGBD Image [52.11275397911693]
  We propose an end-to-end trainable, cross-category method for reconstructing multiple man-made articulated objects from a single RGBD image.<n>We depart from previous works that rely on learning instance-level latent space, focusing on man-made articulated objects with predefined part counts.<n>Our method successfully reconstructs variously structured multiple instances that previous works cannot handle, and outperforms prior works in shape reconstruction and kinematics estimation.
  arXiv  Detail & Related papers  (2025-04-04T05:08:04Z)
• Geometric Point Attention Transformer for 3D Shape Reassembly [17.34739330880715]
  We present a network specifically designed to address the challenges of reasoning about geometric relationships.<n>We integrate both global shape information and local pairwise geometric features, along with poses represented as rotation and translation vectors for each part.<n>We evaluate our model on both the semantic and geometric assembly tasks, showing that it outperforms previous methods in absolute pose estimation.
  arXiv  Detail & Related papers  (2024-11-26T15:29:38Z)
• 3D Geometric Shape Assembly via Efficient Point Cloud Matching [59.241448711254485]
  We introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable matching between mating surfaces of parts. Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task. We evaluate the proposed PMTR on the large-scale 3D geometric shape assembly benchmark dataset of Breaking Bad.
  arXiv  Detail & Related papers  (2024-07-15T08:50:02Z)
• Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers [24.89380678499307]
  We develop a scalable framework for geometric fracture assembly without relying on semantic information. We introduce a novel loss function, i.e., the geometric-based collision loss, to address collision issues during the fracture assembly process. Our framework exhibits better performance on both PartNet and Breaking Bad datasets compared to existing state-of-the-art frameworks.
  arXiv  Detail & Related papers  (2023-12-19T17:13:51Z)
• Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images [71.91424164693422]
  We introduce an explicit point-based human reconstruction framework called HaP.<n>Our approach is featured by fully-explicit point cloud estimation, manipulation, generation, and refinement in the 3D geometric space.<n>Our results may indicate a paradigm rollback to the fully-explicit and geometry-centric algorithm design.
  arXiv  Detail & Related papers  (2023-11-06T05:52:29Z)
• Generalized Few-Shot Point Cloud Segmentation Via Geometric Words [54.32239996417363]
  Few-shot point cloud segmentation algorithms learn to adapt to new classes at the sacrifice of segmentation accuracy for the base classes. We present the first attempt at a more practical paradigm of generalized few-shot point cloud segmentation. We propose the geometric words to represent geometric components shared between the base and novel classes, and incorporate them into a novel geometric-aware semantic representation.
  arXiv  Detail & Related papers  (2023-09-20T11:24:33Z)
• Category-Level Multi-Part Multi-Joint 3D Shape Assembly [36.74814134087434]
  We propose a hierarchical graph learning approach composed of two levels of graph representation learning. The part graph takes part geometries as input to build the desired shape structure. The joint-level graph uses part joints information and focuses on matching and aligning joints.
  arXiv  Detail & Related papers  (2023-03-10T19:02:26Z)
• 3D Part Assembly Generation with Instance Encoded Transformer [22.330218525999857]
  We propose a multi-layer transformer-based framework that involves geometric and relational reasoning between parts to update the part poses iteratively. We extend our framework to a new task called in-process part assembly. Our method achieves far more than 10% improvements over the current state-of-the-art in multiple metrics on the public PartNet dataset.
  arXiv  Detail & Related papers  (2022-07-05T02:40:57Z)
• Learning to Assemble Geometric Shapes [40.2815083025929]
  Assembling parts into an object is a problem that arises in a variety of contexts in the real world and involves numerous applications in science and engineering. Previous work tackles limited cases with identical unit parts or jigsaw-style parts of textured shapes. In this work, we introduce the more challenging problem of shape assembly, which involves textureless fragments of arbitrary shapes with indistinctive junctions.
  arXiv  Detail & Related papers  (2022-05-24T06:07:13Z)
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [50.22269760171131]
  The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. This text is concerned with exposing pre-defined regularities through unified geometric principles. It provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers.
  arXiv  Detail & Related papers  (2021-04-27T21:09:51Z)
• Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image [102.44347847154867]
  We propose a novel formulation that allows to jointly recover the geometry of a 3D object as a set of primitives. Our model recovers the higher level structural decomposition of various objects in the form of a binary tree of primitives. Our experiments on the ShapeNet and D-FAUST datasets demonstrate that considering the organization of parts indeed facilitates reasoning about 3D geometry.
  arXiv  Detail & Related papers  (2020-04-02T17:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.