Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform

• URL: http://arxiv.org/abs/2404.11156v2
• Date: Sat, 20 Apr 2024 18:10:34 GMT
• Title: Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform
• Authors: Chunghyun Park, Seungwook Kim, Jaesik Park, Minsu Cho,
• Abstract summary: We introduce a novel self-supervised Rotation-Invariant 3D correspondence learner with Local Shape Transform, dubbed RIST. RIST learns to establish dense correspondences between shapes even under challenging intra-class variations and arbitrary orientations. RIST demonstrates state-of-the-art performances on 3D part label transfer and semantic keypoint transfer given arbitrarily rotated point cloud pairs.
• Score: 62.27337227010514
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Establishing accurate 3D correspondences between shapes stands as a pivotal challenge with profound implications for computer vision and robotics. However, existing self-supervised methods for this problem assume perfect input shape alignment, restricting their real-world applicability. In this work, we introduce a novel self-supervised Rotation-Invariant 3D correspondence learner with Local Shape Transform, dubbed RIST, that learns to establish dense correspondences between shapes even under challenging intra-class variations and arbitrary orientations. Specifically, RIST learns to dynamically formulate an SO(3)-invariant local shape transform for each point, which maps the SO(3)-equivariant global shape descriptor of the input shape to a local shape descriptor. These local shape descriptors are provided as inputs to our decoder to facilitate point cloud self- and cross-reconstruction. Our proposed self-supervised training pipeline encourages semantically corresponding points from different shapes to be mapped to similar local shape descriptors, enabling RIST to establish dense point-wise correspondences. RIST demonstrates state-of-the-art performances on 3D part label transfer and semantic keypoint transfer given arbitrarily rotated point cloud pairs, outperforming existing methods by significant margins.

Related papers

• Rethinking Rotation Invariance with Point Cloud Registration [18.829454172955202]
  We propose an effective framework for rotation invariance learning via three sequential stages, namely rotation-invariant shape encoding, aligned feature integration, and deep feature registration. Experimental results on 3D shape classification, part segmentation, and retrieval tasks prove the feasibility of our work.
  arXiv  Detail & Related papers  (2022-12-31T08:17:09Z)
• ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes [55.689763519293464]
  ConDor is a self-supervised method that learns to canonicalize the 3D orientation and position for full and partial 3D point clouds. During inference, our method takes an unseen full or partial 3D point cloud at an arbitrary pose and outputs an equivariant canonical pose.
  arXiv  Detail & Related papers  (2022-01-19T18:57:21Z)
• 3D Unsupervised Region-Aware Registration Transformer [13.137287695912633]
  Learning robust point cloud registration models with deep neural networks has emerged as a powerful paradigm. We propose a new design of 3D region partition module that is able to divide the input shape to different regions with a self-supervised 3D shape reconstruction loss. Our experiments show that our 3D-URRT achieves superior registration performance over various benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-07T15:06:52Z)
• Learning 3D Dense Correspondence via Canonical Point Autoencoder [108.20735652143787]
  We propose a canonical point autoencoder (CPAE) that predicts dense correspondences between 3D shapes of the same category. The autoencoder performs two key functions: (a) encoding an arbitrarily ordered point cloud to a canonical primitive, and (b) decoding the primitive back to the original input instance shape.
  arXiv  Detail & Related papers  (2021-07-10T15:54:48Z)
• Rotation-Invariant Local-to-Global Representation Learning for 3D Point Cloud [42.86112554931754]
  We propose a local-to-global representation learning algorithm for 3D point cloud data. Our model takes advantage of multi-level abstraction based on graph convolutional neural networks. The proposed algorithm presents the state-of-the-art performance on the rotation-augmented 3D object recognition and segmentation benchmarks.
  arXiv  Detail & Related papers  (2020-10-07T10:30:20Z)
• Learning Local Neighboring Structure for Robust 3D Shape Representation [143.15904669246697]
  Representation learning for 3D meshes is important in many computer vision and graphics applications. We propose a local structure-aware anisotropic convolutional operation (LSA-Conv) Our model produces significant improvement in 3D shape reconstruction compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-04-21T13:40:03Z)
• A Rotation-Invariant Framework for Deep Point Cloud Analysis [132.91915346157018]
  We introduce a new low-level purely rotation-invariant representation to replace common 3D Cartesian coordinates as the network inputs. Also, we present a network architecture to embed these representations into features, encoding local relations between points and their neighbors, and the global shape structure. We evaluate our method on multiple point cloud analysis tasks, including shape classification, part segmentation, and shape retrieval.
  arXiv  Detail & Related papers  (2020-03-16T14:04:45Z)
• Quaternion Equivariant Capsule Networks for 3D Point Clouds [58.566467950463306]
  We present a 3D capsule module for processing point clouds that is equivariant to 3D rotations and translations. We connect dynamic routing between capsules to the well-known Weiszfeld algorithm. Based on our operator, we build a capsule network that disentangles geometry from pose.
  arXiv  Detail & Related papers  (2019-12-27T13:51:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.