Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

• URL: http://arxiv.org/abs/2104.08160v1
• Date: Fri, 16 Apr 2021 15:16:09 GMT
• Title: Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration
• Authors: Shaofei Wang, Andreas Geiger, Siyu Tang
• Abstract summary: We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space. We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
• Score: 67.69257782645789
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Registering point clouds of dressed humans to parametric human models is a challenging task in computer vision. Traditional approaches often rely on heavily engineered pipelines that require accurate manual initialization of human poses and tedious post-processing. More recently, learning-based methods are proposed in hope to automate this process. We observe that pose initialization is key to accurate registration but existing methods often fail to provide accurate pose initialization. One major obstacle is that, regressing joint rotations from point clouds or images of humans is still very challenging. To this end, we propose novel piecewise transformation fields (PTF), a set of functions that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space. We combine PTF with multi-class occupancy networks, obtaining a novel learning-based framework that learns to simultaneously predict shape and per-point correspondences between the posed space and the canonical space for clothed human. Our key insight is that the translation vector for each query point can be effectively estimated using the point-aligned local features; consequently, rigid per bone transformations and joint rotations can be obtained efficiently via a least-square fitting given the estimated point correspondences, circumventing the challenging task of directly regressing joint rotations from neural networks. Furthermore, the proposed PTF facilitate canonicalized occupancy estimation, which greatly improves generalization capability and results in more accurate surface reconstruction with only half of the parameters compared with the state-of-the-art. Both qualitative and quantitative studies show that fitting parametric models with poses initialized by our network results in much better registration quality, especially for extreme poses.

Related papers

• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• Pose Priors from Language Models [74.61186408764559]
  We present a zero-shot pose optimization method that enforces accurate physical contact constraints. Our method produces surprisingly compelling pose reconstructions of people in close contact. Unlike previous approaches, our method provides a unified framework for resolving self-contact and person-to-person contact.
  arXiv  Detail & Related papers  (2024-05-06T17:59:36Z)
• Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance [48.39751410262664]
  ArtEq is a part-based SE(3)-equivariant neural architecture for SMPL model estimation from point clouds. Experimental results show that ArtEq generalizes to poses not seen during training, outperforming state-of-the-art methods by 44% in terms of body reconstruction accuracy.
  arXiv  Detail & Related papers  (2023-04-20T17:58:26Z)
• OReX: Object Reconstruction from Planar Cross-sections Using Neural Fields [10.862993171454685]
  OReX is a method for 3D shape reconstruction from slices alone, featuring a Neural Field gradients as the prior. A modest neural network is trained on the input planes to return an inside/outside estimate for a given 3D coordinate, yielding a powerful prior that induces smoothness and self-similarities. We offer an iterative estimation architecture and a hierarchical input sampling scheme that encourage coarse-to-fine training, allowing the training process to focus on high frequencies at later stages.
  arXiv  Detail & Related papers  (2022-11-23T11:44:35Z)
• Back to MLP: A Simple Baseline for Human Motion Prediction [59.18776744541904]
  This paper tackles the problem of human motion prediction, consisting in forecasting future body poses from historically observed sequences. We show that the performance of these approaches can be surpassed by a light-weight and purely architectural architecture with only 0.14M parameters. An exhaustive evaluation on Human3.6M, AMASS and 3DPW datasets shows that our method, which we dub siMLPe, consistently outperforms all other approaches.
  arXiv  Detail & Related papers  (2022-07-04T16:35:58Z)
• DFC: Deep Feature Consistency for Robust Point Cloud Registration [0.4724825031148411]
  We present a novel learning-based alignment network for complex alignment scenes. We validate our approach on the 3DMatch dataset and the KITTI odometry dataset.
  arXiv  Detail & Related papers  (2021-11-15T08:27:21Z)
• Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation [75.59415852802958]
  Shape-My-Face (SMF) is a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model. Our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets.
  arXiv  Detail & Related papers  (2020-12-16T20:02:36Z)
• Unsupervised Learning of 3D Point Set Registration [15.900382629390297]
  Point cloud registration is the process of aligning a pair of point sets via searching for a geometric transformation. This paper proposes Deep-3DAligner, a novel unsupervised registration framework based on a newly introduced deep Spatial Correlation Representation (SCR) feature. Our method starts with optimizing a randomly latent SCR feature, which is then decoded to a geometric transformation to align source and target point sets.
  arXiv  Detail & Related papers  (2020-06-11T05:21:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.