Unsupervised 3D Human Mesh Recovery from Noisy Point Clouds

• URL: http://arxiv.org/abs/2107.07539v1
• Date: Thu, 15 Jul 2021 18:07:47 GMT
• Title: Unsupervised 3D Human Mesh Recovery from Noisy Point Clouds
• Authors: Xinxin Zuo and Sen Wang and Minglun Gong and Li Cheng
• Abstract summary: We present an unsupervised approach to reconstruct human shape and pose from noisy point cloud. Our network is trained from scratch with no need to warm-up the network with supervised data.
• Score: 30.401088478228235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a novel unsupervised approach to reconstruct human shape and pose from noisy point cloud. Traditional approaches search for correspondences and conduct model fitting iteratively where a good initialization is critical. Relying on large amount of dataset with ground-truth annotations, recent learning-based approaches predict correspondences for every vertice on the point cloud; Chamfer distance is usually used to minimize the distance between a deformed template model and the input point cloud. However, Chamfer distance is quite sensitive to noise and outliers, thus could be unreliable to assign correspondences. To address these issues, we model the probability distribution of the input point cloud as generated from a parametric human model under a Gaussian Mixture Model. Instead of explicitly aligning correspondences, we treat the process of correspondence search as an implicit probabilistic association by updating the posterior probability of the template model given the input. A novel unsupervised loss is further derived that penalizes the discrepancy between the deformed template and the input point cloud conditioned on the posterior probability. Our approach is very flexible, which works with both complete point cloud and incomplete ones including even a single depth image as input. Our network is trained from scratch with no need to warm-up the network with supervised data. Compared to previous unsupervised methods, our method shows the capability to deal with substantial noise and outliers. Extensive experiments conducted on various public synthetic datasets as well as a very noisy real dataset (i.e. CMU Panoptic) demonstrate the superior performance of our approach over the state-of-the-art methods. Code can be found \url{https://github.com/wangsen1312/unsupervised3dhuman.git}

Related papers

• Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models [19.795578581043745]
  Test-time adaptation of 3D point clouds is crucial for mitigating discrepancies between training and testing samples in real-world scenarios. We introduce a novel 3D test-time adaptation method, termed 3DD-TTA, which stands for 3D Denoising Diffusion Test-Time Adaptation.
  arXiv  Detail & Related papers  (2024-11-21T00:04:38Z)
• Sub-graph Based Diffusion Model for Link Prediction [43.15741675617231]
  Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities. We build a novel generative model for link prediction using a dedicated design to decompose the likelihood estimation process via the Bayesian formula. Our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-13T02:23:55Z)
• Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network [22.81108868492533]
  We propose a 3D anomaly synthesis pipeline to adapt existing large-scale 3Dmodels for 3D anomaly detection. Anomaly-ShapeNet consists of 1600 point cloud samples under 40 categories, which provides a rich and varied collection of data. We also propose a self-supervised method, i.e., Iterative Mask Reconstruction Network (IMRNet), to enable scalable representation learning for 3D anomaly localization.
  arXiv  Detail & Related papers  (2023-11-25T01:45:09Z)
• Mixing-Denoising Generalizable Occupancy Networks [10.316008740970037]
  Current state-of-the-art implicit neural shape models rely on the inductive bias of convolutions. We relax the intrinsic model bias and constrain the hypothesis space instead with an auxiliary regularization related to the reconstruction task. The resulting model is the first only-MLP locally conditioned reconstruction from point cloud network.
  arXiv  Detail & Related papers  (2023-11-20T19:05:57Z)
• Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation [79.60988242843437]
  We propose a novel approach that achieves self-supervised and magnification-flexible point clouds upsampling simultaneously. Experimental results demonstrate that our self-supervised learning based scheme achieves competitive or even better performance than supervised learning based state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-18T07:18:25Z)
• DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction [29.191330510706408]
  We present a new method for real-time non-rigid dense correspondence between point clouds based on structured shape construction. Our method, termed Deep Point Correspondence (DPC), requires a fraction of the training data compared to previous techniques. Our construction scheme leads to a performance boost in comparison to recent state-of-the-art correspondence methods.
  arXiv  Detail & Related papers  (2021-10-16T18:41:13Z)
• Probabilistic Modeling for Human Mesh Recovery [73.11532990173441]
  This paper focuses on the problem of 3D human reconstruction from 2D evidence. We recast the problem as learning a mapping from the input to a distribution of plausible 3D poses.
  arXiv  Detail & Related papers  (2021-08-26T17:55:11Z)
• Variational Bayesian Unlearning [54.26984662139516]
  We study the problem of approximately unlearning a Bayesian model from a small subset of the training data to be erased. We show that it is equivalent to minimizing an evidence upper bound which trades off between fully unlearning from erased data vs. not entirely forgetting the posterior belief. In model training with VI, only an approximate (instead of exact) posterior belief given the full data can be obtained, which makes unlearning even more challenging.
  arXiv  Detail & Related papers  (2020-10-24T11:53:00Z)
• LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration [123.62341095156611]
  LoopReg is an end-to-end learning framework to register a corpus of scans to a common 3D human model. A backward map, parameterized by a Neural Network, predicts the correspondence from every scan point to the surface of the human model. A forward map, parameterized by a human model, transforms the corresponding points back to the scan based on the model parameters (pose and shape)
  arXiv  Detail & Related papers  (2020-10-23T14:39:50Z)
• Refinement of Predicted Missing Parts Enhance Point Cloud Completion [62.997667081978825]
  Point cloud completion is the task of predicting complete geometry from partial observations using a point set representation for a 3D shape. Previous approaches propose neural networks to directly estimate the whole point cloud through encoder-decoder models fed by the incomplete point set. This paper proposes an end-to-end neural network architecture that focuses on computing the missing geometry and merging the known input and the predicted point cloud.
  arXiv  Detail & Related papers  (2020-10-08T22:01:23Z)
• Learning Nonparametric Human Mesh Reconstruction from a Single Image without Ground Truth Meshes [56.27436157101251]
  We propose a novel approach to learn human mesh reconstruction without any ground truth meshes. This is made possible by introducing two new terms into the loss function of a graph convolutional neural network (Graph CNN)
  arXiv  Detail & Related papers  (2020-02-28T20:30:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.