VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds

• URL: http://arxiv.org/abs/2110.08729v1
• Date: Sun, 17 Oct 2021 05:42:04 GMT
• Title: VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds
• Authors: Guanze Liu, Yu Rong, Lu Sheng
• Abstract summary: We make the first attempt to reconstruct reliable 3D human shapes from single-frame partial point clouds. We propose an end-to-end learnable method, named VoteHMR. The proposed method achieves state-of-the-art performances on two large-scale datasets.
• Score: 32.72878775887121
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: 3D human mesh recovery from point clouds is essential for various tasks, including AR/VR and human behavior understanding. Previous works in this field either require high-quality 3D human scans or sequential point clouds, which cannot be easily applied to low-quality 3D scans captured by consumer-level depth sensors. In this paper, we make the first attempt to reconstruct reliable 3D human shapes from single-frame partial point clouds.To achieve this, we propose an end-to-end learnable method, named VoteHMR. The core of VoteHMR is a novel occlusion-aware voting network that can first reliably produce visible joint-level features from the input partial point clouds, and then complete the joint-level features through the kinematic tree of the human skeleton. Compared with holistic features used by previous works, the joint-level features can not only effectively encode the human geometry information but also be robust to noisy inputs with self-occlusions and missing areas. By exploiting the rich complementary clues from the joint-level features and global features from the input point clouds, the proposed method encourages reliable and disentangled parameter predictions for statistical 3D human models, such as SMPL. The proposed method achieves state-of-the-art performances on two large-scale datasets, namely SURREAL and DFAUST. Furthermore, VoteHMR also demonstrates superior generalization ability on real-world datasets, such as Berkeley MHAD.

Related papers

• HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset [6.978870586488504]
  We present HumanRig, the first large-scale dataset specifically designed for 3D humanoid character rigging. We introduce an innovative, data-driven automatic rigging framework, which overcomes the limitations of GNN-based methods. This work not only remedies the dataset deficiency in rigging research but also propels the animation industry towards more efficient and automated character rigging pipelines.
  arXiv  Detail & Related papers  (2024-12-03T09:33:00Z)
• PointDifformer: Robust Point Cloud Registration With Neural Diffusion and Transformer [31.02661827570958]
  Point cloud registration is a fundamental technique in 3-D computer vision with applications in graphics, autonomous driving, and robotics. We propose a robust point cloud registration approach that leverages graph neural partial differential equations (PDEs) and heat kernel signatures. Empirical experiments on a 3-D point cloud dataset demonstrate that our approach not only achieves state-of-the-art performance for point cloud registration but also exhibits better robustness to additive noise or 3-D shape perturbations.
  arXiv  Detail & Related papers  (2024-04-22T09:50:12Z)
• FILP-3D: Enhancing 3D Few-shot Class-incremental Learning with Pre-trained Vision-Language Models [59.13757801286343]
  Few-shot class-incremental learning aims to mitigate the catastrophic forgetting issue when a model is incrementally trained on limited data. We introduce the FILP-3D framework with two novel components: the Redundant Feature Eliminator (RFE) for feature space misalignment and the Spatial Noise Compensator (SNC) for significant noise.
  arXiv  Detail & Related papers  (2023-12-28T14:52:07Z)
• Sampling is Matter: Point-guided 3D Human Mesh Reconstruction [0.0]
  This paper presents a simple yet powerful method for 3D human mesh reconstruction from a single RGB image. Experimental results on benchmark datasets show that the proposed method efficiently improves the performance of 3D human mesh reconstruction.
  arXiv  Detail & Related papers  (2023-04-19T08:45:26Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• Blind Quality Assessment of 3D Dense Point Clouds with Structure Guided Resampling [71.68672977990403]
  We propose an objective point cloud quality index with Structure Guided Resampling (SGR) to automatically evaluate the perceptually visual quality of 3D dense point clouds. The proposed SGR is a general-purpose blind quality assessment method without the assistance of any reference information.
  arXiv  Detail & Related papers  (2022-08-31T02:42:55Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• Neural Capture of Animatable 3D Human from Monocular Video [38.974181971541846]
  We present a novel paradigm of building an animatable 3D human representation from a monocular video input, such that it can be rendered in any unseen poses and views. Our method is based on a dynamic Neural Radiance Field (NeRF) rigged by a mesh-based parametric 3D human model serving as a geometry proxy.
  arXiv  Detail & Related papers  (2022-08-18T09:20:48Z)
• Higher-Order Implicit Fairing Networks for 3D Human Pose Estimation [1.1501261942096426]
  We introduce a higher-order graph convolutional framework with initial residual connections for 2D-to-3D pose estimation. Our model is able to capture the long-range dependencies between body joints. Experiments and ablations studies conducted on two standard benchmarks demonstrate the effectiveness of our model.
  arXiv  Detail & Related papers  (2021-11-01T13:48:55Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)
• A Nearest Neighbor Network to Extract Digital Terrain Models from 3D Point Clouds [1.6249267147413524]
  We present an algorithm that operates on 3D-point clouds and estimates the underlying DTM for the scene using an end-to-end approach. Our model learns neighborhood information and seamlessly integrates this with point-wise and block-wise global features.
  arXiv  Detail & Related papers  (2020-05-21T15:54:55Z)
• HEMlets PoSh: Learning Part-Centric Heatmap Triplets for 3D Human Pose and Shape Estimation [60.35776484235304]
  This work attempts to address the uncertainty of lifting the detected 2D joints to the 3D space by introducing an intermediate state-Part-Centric Heatmap Triplets (HEMlets) The HEMlets utilize three joint-heatmaps to represent the relative depth information of the end-joints for each skeletal body part. A Convolutional Network (ConvNet) is first trained to predict HEMlets from the input image, followed by a volumetric joint-heatmap regression.
  arXiv  Detail & Related papers  (2020-03-10T04:03:45Z)
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features [51.04841465193678]
  We leverage a 3D fully convolutional network for 3D point clouds. We propose a novel and practical learning mechanism that densely predicts both a detection score and a description feature for each 3D point. Our method achieves state-of-the-art results in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2020-03-06T12:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.