Pix2Repair: Implicit Shape Restoration from Images

• URL: http://arxiv.org/abs/2305.18273v3
• Date: Wed, 20 Dec 2023 05:00:02 GMT
• Title: Pix2Repair: Implicit Shape Restoration from Images
• Authors: Xinchao Song, Nikolas Lamb, Sean Banerjee, Natasha Kholgade Banerjee
• Abstract summary: Pix2Repair takes an image of a fractured object as input and automatically generates a 3D printable restoration shape. We also introduce Fantastic Breaks Imaged, the first large-scale dataset of 11,653 real-world images of fractured objects.
• Score: 7.663519916453075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Pix2Repair, an automated shape repair approach that generates restoration shapes from images to repair fractured objects. Prior repair approaches require a high-resolution watertight 3D mesh of the fractured object as input. Input 3D meshes must be obtained using expensive 3D scanners, and scanned meshes require manual cleanup, limiting accessibility and scalability. Pix2Repair takes an image of the fractured object as input and automatically generates a 3D printable restoration shape. We contribute a novel shape function that deconstructs a latent code representing the fractured object into a complete shape and a break surface. We also introduce Fantastic Breaks Imaged, the first large-scale dataset of 11,653 real-world images of fractured objects for training and evaluating image-based shape repair approaches. Our dataset contains images of objects from Fantastic Breaks, complete with rich annotations. We show restorations for real fractures from our dataset, and for synthetic fractures from the Geometric Breaks and Breaking Bad datasets. Our approach outperforms shape completion approaches adapted for shape repair in terms of chamfer distance, normal consistency, and percent restorations generated.

Related papers

• Fantastic Breaks: A Dataset of Paired 3D Scans of Real-World Broken Objects and Their Complete Counterparts [0.5572870549559665]
  We present Fantastic Breaks, a dataset containing scanned, waterproofed, and cleaned 3D meshes for 150 broken objects. Fantastic Breaks contains class and material labels, proxy repair parts that join to broken meshes, and manually annotated fracture boundaries. We show experimental shape repair evaluation with Fantastic Breaks using multiple learning-based approaches.
  arXiv  Detail & Related papers  (2023-03-24T17:03:40Z)
• DeepJoin: Learning a Joint Occupancy, Signed Distance, and Normal Field Function for Shape Repair [0.684225774857327]
  DeepJoin is an automated approach to generate high-resolution repairs for fractured shapes using deep neural networks. We present a novel implicit shape representation for fractured shape repair that combines the occupancy function, signed distance function, and normal field.
  arXiv  Detail & Related papers  (2022-11-22T16:44:57Z)
• DeepMend: Learning Occupancy Functions to Represent Shape for Repair [0.6087960723103347]
  DeepMend is a novel approach to reconstruct restorations to fractured shapes using learned occupancy functions. We represent the occupancy of a fractured shape as the conjunction of the occupancy of an underlying complete shape and the fracture surface. We show results with simulated fractures on synthetic and real-world scanned objects, and with scanned real fractured mugs.
  arXiv  Detail & Related papers  (2022-10-11T18:42:20Z)
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape [77.95154911528365]
  3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. Previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry. This paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person.
  arXiv  Detail & Related papers  (2022-04-09T03:46:18Z)
• Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian [58.704089101826774]
  We present a 3D-aware image deformation method with minimal restrictions on shape category and deformation type. We take a supervised learning-based approach to predict the shape Laplacian of the underlying volume of a 3D reconstruction represented as a point cloud. In the experiments, we present our results of deforming 2D character and clothed human images.
  arXiv  Detail & Related papers  (2022-03-29T04:57:18Z)
• Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image [26.464091507125826]
  We introduce a template-based method to infer 3D shapes from a single-view image and apply the reconstructed mesh to a downstream task. Our method faithfully reconstructs 3D meshes and achieves state-of-the-art accuracy in a length measurement task on a severely deformed fish dataset.
  arXiv  Detail & Related papers  (2022-01-23T21:46:30Z)
• Object Wake-up: 3-D Object Reconstruction, Animation, and in-situ Rendering from a Single Image [58.69732754597448]
  Given a picture of a chair, could we extract the 3-D shape of the chair, animate its plausible articulations and motions, and render in-situ in its original image space? We devise an automated approach to extract and manipulate articulated objects in single images.
  arXiv  Detail & Related papers  (2021-08-05T16:20:12Z)
• From Points to Multi-Object 3D Reconstruction [71.17445805257196]
  We propose a method to detect and reconstruct multiple 3D objects from a single RGB image. A keypoint detector localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes. The presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable.
  arXiv  Detail & Related papers  (2020-12-21T18:52:21Z)
• 3DBooSTeR: 3D Body Shape and Texture Recovery [76.91542440942189]
  3DBooSTeR is a novel method to recover a textured 3D body mesh from a partial 3D scan. The proposed approach decouples the shape and texture completion into two sequential tasks.
  arXiv  Detail & Related papers  (2020-10-23T21:07:59Z)
• AutoSweep: Recovering 3D Editable Objectsfrom a Single Photograph [54.701098964773756]
  We aim to recover 3D objects with semantic parts and can be directly edited. Our work makes an attempt towards recovering two types of primitive-shaped objects, namely, generalized cuboids and generalized cylinders. Our algorithm can recover high quality 3D models and outperforms existing methods in both instance segmentation and 3D reconstruction.
  arXiv  Detail & Related papers  (2020-05-27T12:16:24Z)
• Single-View 3D Object Reconstruction from Shape Priors in Memory [15.641803721287628]
  Existing methods for single-view 3D object reconstruction do not contain enough information to reconstruct high-quality 3D shapes. We propose a novel method, named Mem3D, that explicitly constructs shape priors to supplement the missing information in the image. We also propose a voxel triplet loss function that helps to retrieve the precise 3D shapes that are highly related to the input image from shape priors.
  arXiv  Detail & Related papers  (2020-03-08T03:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.