Jigsaw: Learning to Assemble Multiple Fractured Objects
- URL: http://arxiv.org/abs/2305.17975v2
- Date: Fri, 27 Oct 2023 03:13:45 GMT
- Title: Jigsaw: Learning to Assemble Multiple Fractured Objects
- Authors: Jiaxin Lu, Yifan Sun, Qixing Huang
- Abstract summary: Jigsaw is a novel framework for assembling physically broken 3D objects from multiple pieces.
We show how to jointly learn segmentation and matching and seamlessly integrate feature matching and rigidity constraints.
Our method also generalizes well to diverse fracture modes, objects, and unseen instances.
- Score: 30.32138466263263
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Automated assembly of 3D fractures is essential in orthopedics, archaeology,
and our daily life. This paper presents Jigsaw, a novel framework for
assembling physically broken 3D objects from multiple pieces. Our approach
leverages hierarchical features of global and local geometry to match and align
the fracture surfaces. Our framework consists of four components: (1) front-end
point feature extractor with attention layers, (2) surface segmentation to
separate fracture and original parts, (3) multi-parts matching to find
correspondences among fracture surface points, and (4) robust global alignment
to recover the global poses of the pieces. We show how to jointly learn
segmentation and matching and seamlessly integrate feature matching and
rigidity constraints. We evaluate Jigsaw on the Breaking Bad dataset and
achieve superior performance compared to state-of-the-art methods. Our method
also generalizes well to diverse fracture modes, objects, and unseen instances.
To the best of our knowledge, this is the first learning-based method designed
specifically for 3D fracture assembly over multiple pieces. Our code is
available at https://jiaxin-lu.github.io/Jigsaw/.
Related papers
- Hierarchically Structured Neural Bones for Reconstructing Animatable Objects from Casual Videos [37.455535904703204]
We propose a new framework for creating and manipulating 3D models of arbitrary objects using casually captured videos.
Our core ingredient is a novel deformation hierarchy model, which captures motions of objects with a tree-structured bones.
Our framework offers several clear advantages: (1) users can obtain animatable 3D models of the arbitrary objects in improved quality from their casual videos, (2) users can manipulate 3D models in an intuitive manner with minimal costs, and (3) users can interactively add or delete control points as necessary.
arXiv Detail & Related papers (2024-08-01T07:42:45Z) - 3D Geometric Shape Assembly via Efficient Point Cloud Matching [59.241448711254485]
We introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable matching between mating surfaces of parts.
Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task.
We evaluate the proposed PMTR on the large-scale 3D geometric shape assembly benchmark dataset of Breaking Bad.
arXiv Detail & Related papers (2024-07-15T08:50:02Z) - Reassembling Broken Objects using Breaking Curves [11.679900805394816]
A robust solution that generalizes well must deal with diverse patterns associated with different types of broken objects.
We propose a method that tackles the pairwise assembly of 3D point clouds, that is agnostic on the type of object.
Results show that our solution performs well in reassembling different kinds of broken objects.
arXiv Detail & Related papers (2023-06-05T11:16:50Z) - Building Rearticulable Models for Arbitrary 3D Objects from 4D Point
Clouds [28.330364666426345]
We build rearticulable models for arbitrary everyday man-made objects containing an arbitrary number of parts.
Our method identifies the distinct object parts, what parts are connected to what other parts, and the properties of the joints connecting each part pair.
arXiv Detail & Related papers (2023-06-01T17:59:21Z) - Fantastic Breaks: A Dataset of Paired 3D Scans of Real-World Broken
Objects and Their Complete Counterparts [0.5572870549559665]
We present Fantastic Breaks, a dataset containing scanned, waterproofed, and cleaned 3D meshes for 150 broken objects.
Fantastic Breaks contains class and material labels, proxy repair parts that join to broken meshes, and manually annotated fracture boundaries.
We show experimental shape repair evaluation with Fantastic Breaks using multiple learning-based approaches.
arXiv Detail & Related papers (2023-03-24T17:03:40Z) - PartAfford: Part-level Affordance Discovery from 3D Objects [113.91774531972855]
We present a new task of part-level affordance discovery (PartAfford)
Given only the affordance labels per object, the machine is tasked to (i) decompose 3D shapes into parts and (ii) discover how each part corresponds to a certain affordance category.
We propose a novel learning framework for PartAfford, which discovers part-level representations by leveraging only the affordance set supervision and geometric primitive regularization.
arXiv Detail & Related papers (2022-02-28T02:58:36Z) - 3D Compositional Zero-shot Learning with DeCompositional Consensus [102.7571947144639]
We argue that part knowledge should be composable beyond the observed object classes.
We present 3D Compositional Zero-shot Learning as a problem of part generalization from seen to unseen object classes.
arXiv Detail & Related papers (2021-11-29T16:34:53Z) - Discovering 3D Parts from Image Collections [98.16987919686709]
We tackle the problem of 3D part discovery from only 2D image collections.
Instead of relying on manually annotated parts for supervision, we propose a self-supervised approach.
Our key insight is to learn a novel part shape prior that allows each part to fit an object shape faithfully while constrained to have simple geometry.
arXiv Detail & Related papers (2021-07-28T20:29:16Z) - AutoSweep: Recovering 3D Editable Objectsfrom a Single Photograph [54.701098964773756]
We aim to recover 3D objects with semantic parts and can be directly edited.
Our work makes an attempt towards recovering two types of primitive-shaped objects, namely, generalized cuboids and generalized cylinders.
Our algorithm can recover high quality 3D models and outperforms existing methods in both instance segmentation and 3D reconstruction.
arXiv Detail & Related papers (2020-05-27T12:16:24Z) - Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from
a Single RGB Image [102.44347847154867]
We propose a novel formulation that allows to jointly recover the geometry of a 3D object as a set of primitives.
Our model recovers the higher level structural decomposition of various objects in the form of a binary tree of primitives.
Our experiments on the ShapeNet and D-FAUST datasets demonstrate that considering the organization of parts indeed facilitates reasoning about 3D geometry.
arXiv Detail & Related papers (2020-04-02T17:58:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.