3D Shape Completion on Unseen Categories:A Weakly-supervised Approach

• URL: http://arxiv.org/abs/2401.10578v2
• Date: Sat, 13 Jul 2024 03:25:01 GMT
• Title: 3D Shape Completion on Unseen Categories:A Weakly-supervised Approach
• Authors: Lintai Wu, Junhui Hou, Linqi Song, Yong Xu,
• Abstract summary: We introduce a novel weakly-supervised framework to reconstruct the complete shapes from unseen categories. We first propose an end-to-end prior-assisted shape learning network that leverages data from the seen categories to infer a coarse shape. In addition, we propose a self-supervised shape refinement model to further refine the coarse shape.
• Score: 61.76304400106871
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D shapes captured by scanning devices are often incomplete due to occlusion. 3D shape completion methods have been explored to tackle this limitation. However, most of these methods are only trained and tested on a subset of categories, resulting in poor generalization to unseen categories. In this paper, we introduce a novel weakly-supervised framework to reconstruct the complete shapes from unseen categories. We first propose an end-to-end prior-assisted shape learning network that leverages data from the seen categories to infer a coarse shape. Specifically, we construct a prior bank consisting of representative shapes from the seen categories. Then, we design a multi-scale pattern correlation module for learning the complete shape of the input by analyzing the correlation between local patterns within the input and the priors at various scales. In addition, we propose a self-supervised shape refinement model to further refine the coarse shape. Considering the shape variability of 3D objects across categories, we construct a category-specific prior bank to facilitate shape refinement. Then, we devise a voxel-based partial matching loss and leverage the partial scans to drive the refinement process. Extensive experimental results show that our approach is superior to state-of-the-art methods by a large margin.

Related papers

• Beyond Complete Shapes: A quantitative Evaluation of 3D Shape Matching Algorithms [41.95394677818476]
  Finding correspondences between 3D shapes is an important problem in computer vision, graphics and beyond. We provide a generic and flexible framework for the procedural generation of challenging partial shape matching scenarios. We manually create cross-dataset correspondences between seven existing (complete geometry) shape matching datasets, leading to a total of 2543 shapes.
  arXiv  Detail & Related papers  (2024-11-05T21:08:19Z)
• Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation [26.982199143972835]
  We introduce a diffusion-driven self-supervised network for multi-object shape reconstruction and categorical pose estimation. Our method significantly outperforms state-of-the-art self-supervised category-level baselines and even surpasses some fully-supervised instance-level and category-level methods.
  arXiv  Detail & Related papers  (2024-03-19T13:43:27Z)
• Geometrically Consistent Partial Shape Matching [50.29468769172704]
  Finding correspondences between 3D shapes is a crucial problem in computer vision and graphics. An often neglected but essential property of matching geometrics is consistency. We propose a novel integer linear programming partial shape matching formulation.
  arXiv  Detail & Related papers  (2023-09-10T12:21:42Z)
• 3D Textured Shape Recovery with Learned Geometric Priors [58.27543892680264]
  This technical report presents our approach to address limitations by incorporating learned geometric priors. We generate a SMPL model from learned pose prediction and fuse it into the partial input to add prior knowledge of human bodies. We also propose a novel completeness-aware bounding box adaptation for handling different levels of scales.
  arXiv  Detail & Related papers  (2022-09-07T16:03:35Z)
• PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories [24.724113526984]
  We propose PatchComplete, which learns effective shape priors based on multi-resolution local patches. Such patch-based priors avoid overfitting to specific train categories and enable reconstruction on entirely unseen categories at test time. We demonstrate the effectiveness of our approach on synthetic ShapeNet data as well as challenging real-scanned objects from ScanNet.
  arXiv  Detail & Related papers  (2022-06-10T07:34:10Z)
• Multi-Category Mesh Reconstruction From Image Collections [90.24365811344987]
  We present an alternative approach that infers the textured mesh of objects combining a series of deformable 3D models and a set of instance-specific deformation, pose, and texture. Our method is trained with images of multiple object categories using only foreground masks and rough camera poses as supervision. Experiments show that the proposed framework can distinguish between different object categories and learn category-specific shape priors in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-21T16:32:31Z)
• Representing Shape Collections with Alignment-Aware Linear Models [17.635846912560627]
  We revisit the classical representation of 3D point clouds as linear shape models. Our key insight is to leverage deep learning to represent a collection of shapes as affine transformations.
  arXiv  Detail & Related papers  (2021-09-03T16:28:34Z)
• Unsupervised 3D Shape Completion through GAN Inversion [116.27680045885849]
  We present ShapeInversion, which introduces Generative Adrial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best fits the given partial input. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data.
  arXiv  Detail & Related papers  (2021-04-27T17:53:46Z)
• Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions [70.0171362989609]
  We propose a novel fine-grained 3D shape classification method named FG3D-Net to capture the fine-grained local details of 3D shapes from multiple rendered views. Our results under the fine-grained 3D shape dataset show that our method outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-05-26T06:53:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.