Shape registration in the time of transformers
- URL: http://arxiv.org/abs/2106.13679v2
- Date: Mon, 28 Jun 2021 07:56:20 GMT
- Title: Shape registration in the time of transformers
- Authors: Giovanni Trappolini, Luca Cosmo, Luca Moschella, Riccardo Marin,
Simone Melzi, Emanuele Rodol\`a
- Abstract summary: We propose a transformer-based procedure for the efficient registration of non-rigid 3D point clouds.
By exploiting the potential of this architecture, we can train our model requiring only a sparse set of ground truth correspondences.
- Score: 9.728331219460287
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we propose a transformer-based procedure for the efficient
registration of non-rigid 3D point clouds. The proposed approach is data-driven
and adopts for the first time the transformer architecture in the registration
task. Our method is general and applies to different settings. Given a fixed
template with some desired properties (e.g. skinning weights or other animation
cues), we can register raw acquired data to it, thereby transferring all the
template properties to the input geometry. Alternatively, given a pair of
shapes, our method can register the first onto the second (or vice-versa),
obtaining a high-quality dense correspondence between the two. In both
contexts, the quality of our results enables us to target real applications
such as texture transfer and shape interpolation. Furthermore, we also show
that including an estimation of the underlying density of the surface eases the
learning process. By exploiting the potential of this architecture, we can
train our model requiring only a sparse set of ground truth correspondences
($10\sim20\%$ of the total points). The proposed model and the analysis that we
perform pave the way for future exploration of transformer-based architectures
for registration and matching applications. Qualitative and quantitative
evaluations demonstrate that our pipeline outperforms state-of-the-art methods
for deformable and unordered 3D data registration on different datasets and
scenarios.
Related papers
- Equi-GSPR: Equivariant SE(3) Graph Network Model for Sparse Point Cloud Registration [2.814748676983944]
We propose a graph neural network model embedded with a local Spherical Euclidean 3D equivariance property through SE(3) message passing based propagation.
Our model is composed mainly of a descriptor module, equivariant graph layers, match similarity, and the final regression layers.
Experiments conducted on the 3DMatch and KITTI datasets exhibit the compelling and robust performance of our model compared to state-of-the-art approaches.
arXiv Detail & Related papers (2024-10-08T06:48:01Z) - Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries.
We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images.
Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
arXiv Detail & Related papers (2024-07-26T06:29:09Z) - Global Point Cloud Registration Network for Large Transformations [46.7301374772952]
We present ReLaTo, an architecture that faces the cases where large transformations happen while maintaining good performance for local transformations.
This paper uses a novel Softmax pooling layer to find correspondences in a bilateral consensus manner between two point sets, sampling the most confident matches.
A target-guided denoising step is then applied to both the obtained matches and latent features, estimating the final fine registration.
arXiv Detail & Related papers (2024-03-26T18:52:48Z) - AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware
Transformers [94.11915008006483]
We present a new method that reformulates point cloud completion as a set-to-set translation problem.
We design a new model, called PoinTr, which adopts a Transformer encoder-decoder architecture for point cloud completion.
Our method attains 6.53 CD on PCN, 0.81 CD on ShapeNet-55 and 0.392 MMD on real-world KITTI.
arXiv Detail & Related papers (2023-01-11T16:14:12Z) - Semantic keypoint-based pose estimation from single RGB frames [64.80395521735463]
We present an approach to estimating the continuous 6-DoF pose of an object from a single RGB image.
The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model.
We show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios.
arXiv Detail & Related papers (2022-04-12T15:03:51Z) - COTReg:Coupled Optimal Transport based Point Cloud Registration [28.730827908402286]
This paper proposes a learning framework COTReg to predict correspondences of 3D point cloud registration.
We transform the two matchings into a Wasserstein distance-based and a Gromov-Wasserstein distance-based optimizations.
Our correspondence prediction pipeline can be easily integrated into either learning-based features like FCGF or traditional descriptors like FPFH.
arXiv Detail & Related papers (2021-12-29T03:20:18Z) - Deep Hough Voting for Robust Global Registration [52.40611370293272]
We present an efficient framework for pairwise registration of real-world 3D scans, leveraging Hough voting in the 6D transformation parameter space.
Our method outperforms state-of-the-art methods on 3DMatch and 3DLoMatch benchmarks while achieving comparable performance on KITTI odometry dataset.
arXiv Detail & Related papers (2021-09-09T14:38:06Z) - U-mesh: Human Correspondence Matching with Mesh Convolutional Networks [15.828285556159026]
We propose an elegant fusion of regression (bottom-up) and generative (top-down) methods to fit a parametric template model to raw scan meshes.
Our first major contribution is an intrinsic convolutional mesh U-net architecture that predicts pointwise correspondence to a template surface.
We evaluate the proposed method on the FAUST correspondence challenge where we achieve 20% (33%) improvement over state of the art methods for inter- (intra-) subject correspondence.
arXiv Detail & Related papers (2021-08-15T08:58:45Z) - Shape My Face: Registering 3D Face Scans by Surface-to-Surface
Translation [75.59415852802958]
Shape-My-Face (SMF) is a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model.
Our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets.
arXiv Detail & Related papers (2020-12-16T20:02:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.