Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images

• URL: http://arxiv.org/abs/2311.02892v1
• Date: Mon, 6 Nov 2023 05:52:29 GMT
• Title: Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images
• Authors: Yingzhi Tang and Qijian Zhang and Junhui Hou and Yebin Liu
• Abstract summary: We introduce an explicit point-based human reconstruction framework called HaP. Our approach is featured by fully-explicit point cloud estimation, manipulation, generation, and refinement in the 3D geometric space. Our results may indicate a paradigm rollback to the fully-explicit and geometry-centric algorithm design.
• Score: 78.56114271538061
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The latest trends in the research field of single-view human reconstruction devote to learning deep implicit functions constrained by explicit body shape priors. Despite the remarkable performance improvements compared with traditional processing pipelines, existing learning approaches still show different aspects of limitations in terms of flexibility, generalizability, robustness, and/or representation capability. To comprehensively address the above issues, in this paper, we investigate an explicit point-based human reconstruction framework called HaP, which adopts point clouds as the intermediate representation of the target geometric structure. Technically, our approach is featured by fully-explicit point cloud estimation, manipulation, generation, and refinement in the 3D geometric space, instead of an implicit learning process that can be ambiguous and less controllable. The overall workflow is carefully organized with dedicated designs of the corresponding specialized learning components as well as processing procedures. Extensive experiments demonstrate that our framework achieves quantitative performance improvements of 20% to 40% over current state-of-the-art methods, and better qualitative results. Our promising results may indicate a paradigm rollback to the fully-explicit and geometry-centric algorithm design, which enables to exploit various powerful point cloud modeling architectures and processing techniques. We will make our code and data publicly available at https://github.com/yztang4/HaP.

Related papers

• Masked Generative Extractor for Synergistic Representation and 3D Generation of Point Clouds [6.69660410213287]
  We propose an innovative framework called Point-MGE to explore the benefits of deeply integrating 3D representation learning and generative learning. In shape classification, Point-MGE achieved an accuracy of 94.2% (+1.0%) on the ModelNet40 dataset and 92.9% (+5.5%) on the ScanObjectNN dataset. Experimental results also confirmed that Point-MGE can generate high-quality 3D shapes in both unconditional and conditional settings.
  arXiv  Detail & Related papers  (2024-06-25T07:57:03Z)
• ParaPoint: Learning Global Free-Boundary Surface Parameterization of 3D Point Clouds [52.03819676074455]
  ParaPoint is an unsupervised neural learning pipeline for achieving global free-boundary surface parameterization. This work makes the first attempt to investigate neural point cloud parameterization that pursues both global mappings and free boundaries.
  arXiv  Detail & Related papers  (2024-03-15T14:35:05Z)
• Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering [93.94371335579321]
  We propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations. Comprehensive experiments underscore our framework's superior generalization capabilities. Our innovative loss functions empower the model to autonomously recover domain-specific scale-and-shift coefficients.
  arXiv  Detail & Related papers  (2023-09-18T12:36:39Z)
• PatchMixer: Rethinking network design to boost generalization for 3D point cloud understanding [2.512827436728378]
  We argue that the ability of a model to transfer the learnt knowledge to different domains is an important feature that should be evaluated to exhaustively assess the quality of a deep network architecture. In this work we propose PatchMixer, a simple yet effective architecture that extends the ideas behind the recent paper to 3D point clouds.
  arXiv  Detail & Related papers  (2023-07-28T17:37:53Z)
• A Systematic Survey in Geometric Deep Learning for Structure-based Drug Design [63.30166298698985]
  Structure-based drug design (SBDD) utilizes the three-dimensional geometry of proteins to identify potential drug candidates. Recent developments in geometric deep learning, focusing on the integration and processing of 3D geometric data, have greatly advanced the field of structure-based drug design.
  arXiv  Detail & Related papers  (2023-06-20T14:21:58Z)
• Towards Confidence-guided Shape Completion for Robotic Applications [6.940242990198]
  Deep learning has begun taking traction as effective means of inferring a complete 3D object representation from partial visual data. We propose an object shape completion method based on an implicit 3D representation providing a confidence value for each reconstructed point. We experimentally validate our approach by comparing reconstructed shapes with ground truths, and by deploying our shape completion algorithm in a robotic grasping pipeline.
  arXiv  Detail & Related papers  (2022-09-09T13:48:24Z)
• Depth Completion using Geometry-Aware Embedding [22.333381291860498]
  This paper proposes an efficient method to learn geometry-aware embedding. It encodes the local and global geometric structure information from 3D points, e.g., scene layout, object's sizes and shapes, to guide dense depth estimation.
  arXiv  Detail & Related papers  (2022-03-21T12:06:27Z)
• Revisiting Point Cloud Simplification: A Learnable Feature Preserving Approach [57.67932970472768]
  Mesh and Point Cloud simplification methods aim to reduce the complexity of 3D models while retaining visual quality and relevant salient features. We propose a fast point cloud simplification method by learning to sample salient points. The proposed method relies on a graph neural network architecture trained to select an arbitrary, user-defined, number of points from the input space and to re-arrange their positions so as to minimize the visual perception error.
  arXiv  Detail & Related papers  (2021-09-30T10:23:55Z)
• Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration [67.69257782645789]
  We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space. We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
  arXiv  Detail & Related papers  (2021-04-16T15:16:09Z)
• Learning Occupancy Function from Point Clouds for Surface Reconstruction [6.85316573653194]
  Implicit function based surface reconstruction has been studied for a long time to recover 3D shapes from point clouds sampled from surfaces. This paper proposes a novel method for learning occupancy functions from sparse point clouds and achieves better performance on challenging surface reconstruction tasks.
  arXiv  Detail & Related papers  (2020-10-22T02:07:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.