GARF: Learning Generalizable 3D Reassembly for Real-World Fractures

• URL: http://arxiv.org/abs/2504.05400v1
• Date: Mon, 07 Apr 2025 18:13:16 GMT
• Title: GARF: Learning Generalizable 3D Reassembly for Real-World Fractures
• Authors: Sihang Li, Zeyu Jiang, Grace Chen, Chenyang Xu, Siqi Tan, Xue Wang, Irving Fang, Kristof Zyskowski, Shannon P. McPherron, Radu Iovita, Chen Feng, Jing Zhang,
• Abstract summary: 3D reassembly is a challenging spatial intelligence task with broad applications across scientific domains.<n>We propose GARF, a generalizable 3D reassembly framework for real-world fractures.<n>In collaboration with archaeologists, paleoanthropologists, and ornithologists, we curate Fractura, a diverse dataset for vision and learning communities.
• Score: 14.531506705859949
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D reassembly is a challenging spatial intelligence task with broad applications across scientific domains. While large-scale synthetic datasets have fueled promising learning-based approaches, their generalizability to different domains is limited. Critically, it remains uncertain whether models trained on synthetic datasets can generalize to real-world fractures where breakage patterns are more complex. To bridge this gap, we propose GARF, a generalizable 3D reassembly framework for real-world fractures. GARF leverages fracture-aware pretraining to learn fracture features from individual fragments, with flow matching enabling precise 6-DoF alignments. At inference time, we introduce one-step preassembly, improving robustness to unseen objects and varying numbers of fractures. In collaboration with archaeologists, paleoanthropologists, and ornithologists, we curate Fractura, a diverse dataset for vision and learning communities, featuring real-world fracture types across ceramics, bones, eggshells, and lithics. Comprehensive experiments have shown our approach consistently outperforms state-of-the-art methods on both synthetic and real-world datasets, achieving 82.87\% lower rotation error and 25.15\% higher part accuracy. This sheds light on training on synthetic data to advance real-world 3D puzzle solving, demonstrating its strong generalization across unseen object shapes and diverse fracture types.

Related papers

• GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency [50.11520458252128]
  Existing 3D affordance learning methods struggle with generalization and robustness due to limited annotated data.<n>We propose GEAL, a novel framework designed to enhance the generalization and robustness of 3D affordance learning by leveraging large-scale pre-trained 2D models.<n>GEAL consistently outperforms existing methods across seen and novel object categories, as well as corrupted data.
  arXiv  Detail & Related papers  (2024-12-12T17:59:03Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• Mixed Diffusion for 3D Indoor Scene Synthesis [55.94569112629208]
  We present MiDiffusion, a novel mixed discrete-continuous diffusion model designed to synthesize plausible 3D indoor scenes.<n>We show it outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis.
  arXiv  Detail & Related papers  (2024-05-31T17:54:52Z)
• REACTO: Reconstructing Articulated Objects from a Single Video [64.89760223391573]
  We propose a novel deformation model that enhances the rigidity of each part while maintaining flexible deformation of the joints. Our method outperforms previous works in producing higher-fidelity 3D reconstructions of general articulated objects.
  arXiv  Detail & Related papers  (2024-04-17T08:01:55Z)
• Leveraging Frequency Domain Learning in 3D Vessel Segmentation [50.54833091336862]
  In this study, we leverage Fourier domain learning as a substitute for multi-scale convolutional kernels in 3D hierarchical segmentation models. We show that our novel network achieves remarkable dice performance (84.37% on ASACA500 and 80.32% on ImageCAS) in tubular vessel segmentation tasks.
  arXiv  Detail & Related papers  (2024-01-11T19:07:58Z)
• Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers [24.89380678499307]
  We develop a scalable framework for geometric fracture assembly without relying on semantic information. We introduce a novel loss function, i.e., the geometric-based collision loss, to address collision issues during the fracture assembly process. Our framework exhibits better performance on both PartNet and Breaking Bad datasets compared to existing state-of-the-art frameworks.
  arXiv  Detail & Related papers  (2023-12-19T17:13:51Z)
• Jigsaw: Learning to Assemble Multiple Fractured Objects [30.32138466263263]
  Jigsaw is a novel framework for assembling physically broken 3D objects from multiple pieces. We show how to jointly learn segmentation and matching and seamlessly integrate feature matching and rigidity constraints. Our method also generalizes well to diverse fracture modes, objects, and unseen instances.
  arXiv  Detail & Related papers  (2023-05-29T09:33:43Z)
• Fantastic Breaks: A Dataset of Paired 3D Scans of Real-World Broken Objects and Their Complete Counterparts [0.5572870549559665]
  We present Fantastic Breaks, a dataset containing scanned, waterproofed, and cleaned 3D meshes for 150 broken objects. Fantastic Breaks contains class and material labels, proxy repair parts that join to broken meshes, and manually annotated fracture boundaries. We show experimental shape repair evaluation with Fantastic Breaks using multiple learning-based approaches.
  arXiv  Detail & Related papers  (2023-03-24T17:03:40Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• MDA GAN: Adversarial-Learning-based 3-D Seismic Data Interpolation and Reconstruction for Complex Missing [6.345037597566314]
  Multi-Dimensional Adrial GAN (MDA GAN) is a novel 3-D GAN framework. MDA GAN employs three discriminators to ensure the consistency of the reconstructed data with the original data distribution in each dimension. The method achieves reasonable reconstructions for up to 95% of random discrete missing, 100 traces of continuous missing and more complex hybrid missing.
  arXiv  Detail & Related papers  (2022-04-07T04:01:53Z)
• Learning perturbation sets for robust machine learning [97.6757418136662]
  We use a conditional generator that defines the perturbation set over a constrained region of the latent space. We measure the quality of our learned perturbation sets both quantitatively and qualitatively. We leverage our learned perturbation sets to train models which are empirically and certifiably robust to adversarial image corruptions and adversarial lighting variations.
  arXiv  Detail & Related papers  (2020-07-16T16:39:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.