ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation

• URL: http://arxiv.org/abs/2311.11106v2
• Date: Mon, 11 Mar 2024 07:06:29 GMT
• Title: ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation
• Authors: Yan Di, Chenyangguang Zhang, Chaowei Wang, Ruida Zhang, Guangyao Zhai, Yanyan Li, Bowen Fu, Xiangyang Ji, Shan Gao
• Abstract summary: We present ShapeMatcher, a unified self-supervised learning framework for joint shape canonicalization, segmentation, retrieval and deformation. The key insight of ShapeMaker is the simultaneous training of the four highly-associated processes: canonicalization, segmentation, retrieval, and deformation.
• Score: 47.94499636697971
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this paper, we present ShapeMatcher, a unified self-supervised learning framework for joint shape canonicalization, segmentation, retrieval and deformation. Given a partially-observed object in an arbitrary pose, we first canonicalize the object by extracting point-wise affine-invariant features, disentangling inherent structure of the object with its pose and size. These learned features are then leveraged to predict semantically consistent part segmentation and corresponding part centers. Next, our lightweight retrieval module aggregates the features within each part as its retrieval token and compare all the tokens with source shapes from a pre-established database to identify the most geometrically similar shape. Finally, we deform the retrieved shape in the deformation module to tightly fit the input object by harnessing part center guided neural cage deformation. The key insight of ShapeMaker is the simultaneous training of the four highly-associated processes: canonicalization, segmentation, retrieval, and deformation, leveraging cross-task consistency losses for mutual supervision. Extensive experiments on synthetic datasets PartNet, ComplementMe, and real-world dataset Scan2CAD demonstrate that ShapeMaker surpasses competitors by a large margin.

Related papers

• KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation [87.23575166061413]
  KP-RED is a unified KeyPoint-driven REtrieval and Deformation framework. It takes object scans as input and jointly retrieves and deforms the most geometrically similar CAD models.
  arXiv  Detail & Related papers  (2024-03-15T08:44:56Z)
• DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation [22.538892330541582]
  We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection. To accommodate structural variations in the collection, our network composes each shape by a selected subset of template parts which are affine-transformed. Our network, coined DAE-Net for Deforming Auto-Encoder, can achieve unsupervised 3D shape co-segmentation that yields fine-grained, compact, and meaningful parts.
  arXiv  Detail & Related papers  (2023-11-22T03:26:07Z)
• Category-level Shape Estimation for Densely Cluttered Objects [94.64287790278887]
  We propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy.
  arXiv  Detail & Related papers  (2023-02-23T13:00:17Z)
• Frame Averaging for Equivariant Shape Space Learning [85.42901997467754]
  A natural way to incorporate symmetries in shape space learning is to ask that the mapping to the shape space (encoder) and mapping from the shape space (decoder) are equivariant to the relevant symmetries. We present a framework for incorporating equivariance in encoders and decoders by introducing two contributions.
  arXiv  Detail & Related papers  (2021-12-03T06:41:19Z)
• NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go [109.88509362837475]
  We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes. NeuroMorph produces smooth and point-to-point correspondences between them. It works well for a large variety of input shapes, including non-isometric pairs from different object categories.
  arXiv  Detail & Related papers  (2021-06-17T12:25:44Z)
• Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds [8.156355030558172]
  Representing complex 3D objects as simple geometric primitives, known as shape abstraction, is important for geometric modeling, structural analysis, and shape synthesis. We propose an unsupervised shape abstraction method to map a point cloud into a compact cuboid representation.
  arXiv  Detail & Related papers  (2021-06-07T09:15:16Z)
• Shape-Oriented Convolution Neural Network for Point Cloud Analysis [59.405388577930616]
  Point cloud is a principal data structure adopted for 3D geometric information encoding. Shape-oriented message passing scheme dubbed ShapeConv is proposed to focus on the representation learning of the underlying shape formed by each local neighboring point.
  arXiv  Detail & Related papers  (2020-04-20T16:11:51Z)
• Cross-Shape Attention for Part Segmentation of 3D Point Clouds [11.437076464287822]
  We propose a cross-shape attention mechanism to enable interactions between a shape's point-wise features and those of other shapes. The mechanism assesses both the degree of interaction between points and also mediates feature propagation across shapes. Our approach yields state-of-the-art results in the popular PartNet dataset.
  arXiv  Detail & Related papers  (2020-03-20T00:23:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.