Super-resolution 3D Human Shape from a Single Low-Resolution Image

• URL: http://arxiv.org/abs/2208.10738v1
• Date: Tue, 23 Aug 2022 05:24:39 GMT
• Title: Super-resolution 3D Human Shape from a Single Low-Resolution Image
• Authors: Marco Pesavento, Marco Volino and Adrian Hilton
• Abstract summary: We propose a novel framework to reconstruct super-resolution human shape from a single low-resolution input image. The proposed framework represents the reconstructed shape with a high-detail implicit function.
• Score: 33.70299493354903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel framework to reconstruct super-resolution human shape from a single low-resolution input image. The approach overcomes limitations of existing approaches that reconstruct 3D human shape from a single image, which require high-resolution images together with auxiliary data such as surface normal or a parametric model to reconstruct high-detail shape. The proposed framework represents the reconstructed shape with a high-detail implicit function. Analogous to the objective of 2D image super-resolution, the approach learns the mapping from a low-resolution shape to its high-resolution counterpart and it is applied to reconstruct 3D shape detail from low-resolution images. The approach is trained end-to-end employing a novel loss function which estimates the information lost between a low and high-resolution representation of the same 3D surface shape. Evaluation for single image reconstruction of clothed people demonstrates that our method achieves high-detail surface reconstruction from low-resolution images without auxiliary data. Extensive experiments show that the proposed approach can estimate super-resolution human geometries with a significantly higher level of detail than that obtained with previous approaches when applied to low-resolution images.

Related papers

• High Resolution Surface Reconstruction of Cultural Heritage Objects Using Shape from Polarization Method [0.0]
  The shape from polarization method has been investigated, a passive method with no drawbacks of active methods. The resolution of the depth maps can be dramatically increased using the information obtained from the polarization light. The fusion of polarization and photogrammetric methods is an appropriate solution for achieving high resolution three-dimensional reconstruction.
  arXiv  Detail & Related papers  (2024-06-21T13:14:48Z)
• Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail [54.03399077258403]
  We present LoD-NeuS, an efficient neural representation for high-frequency geometry detail recovery and anti-aliased novel view rendering. Our representation aggregates space features from a multi-convolved featurization within a conical frustum along a ray.
  arXiv  Detail & Related papers  (2023-09-19T05:44:00Z)
• Disjoint Pose and Shape for 3D Face Reconstruction [4.096453902709292]
  We propose an end-to-end pipeline that disjointly solves for pose and shape to make the optimization stable and accurate. The proposed method achieves end-to-end topological consistency, enables iterative face pose refinement procedure, and show remarkable improvement on both quantitative and qualitative results.
  arXiv  Detail & Related papers  (2023-08-26T15:18:32Z)
• High-Resolution Volumetric Reconstruction for Clothed Humans [27.900514732877827]
  We present a novel method for reconstructing clothed humans from a sparse set of, e.g., 1 to 6 RGB images. Our method significantly reduces the mean point-to-surface (P2S) precision of state-of-the-art methods by more than 50% to achieve approximately 2mm accuracy with a 512 volume resolution.
  arXiv  Detail & Related papers  (2023-07-25T06:37:50Z)
• SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery [54.64663713249079]
  SIDER is a novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner. In contrast to prior work, SIDER does not rely on any dataset priors and does not require additional supervision from multiple views, lighting changes or ground truth 3D shape.
  arXiv  Detail & Related papers  (2021-08-11T22:34:53Z)
• 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos [107.36352212367179]
  We propose RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. The proposed method is able to learn 3D body pose and shape across different resolutions with one single model. We extend the RSC-Net to handle low-resolution videos and apply it to reconstruct textured 3D pedestrians from low-resolution input.
  arXiv  Detail & Related papers  (2021-03-11T06:52:12Z)
• Detailed 3D Human Body Reconstruction from Multi-view Images Combining Voxel Super-Resolution and Learned Implicit Representation [12.459968574683625]
  We propose a coarse-to-fine method to reconstruct a detailed 3D human body from multi-view images. The coarse 3D models are estimated by learning an implicit representation based on multi-scale features. The refined detailed 3D human body models can be produced by the voxel super-resolution which can preserve the details.
  arXiv  Detail & Related papers  (2020-12-11T08:07:39Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)
• 3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning [105.49950571267715]
  Existing deep learning methods for 3D human shape and pose estimation rely on relatively high-resolution input images. We propose RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. We show that both these new training losses provide robustness when learning 3D shape and pose in a weakly-supervised manner.
  arXiv  Detail & Related papers  (2020-07-27T16:19:52Z)
• PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction [67.08350202974434]
  We propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function. We show that our method achieves state-of-the-art performance for image-based 3D human reconstruction in the cases of challenging poses and clothing types.
  arXiv  Detail & Related papers  (2020-07-08T02:26:19Z)
• PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization [38.6438956631446]
  Recent advances in image-based 3D human shape estimation have been driven by the significant improvement in representation power afforded by deep neural networks. We argue that this limitation stems primarily form two conflicting requirements; accurate predictions require large context, but precise predictions require high resolution. We formulate a multi-level architecture that is end-to-end trainable. A coarse level observes the whole image at lower resolution and focuses on holistic reasoning. We demonstrate that our approach significantly outperforms existing state-of-the-art techniques on single image human shape reconstruction by fully leveraging 1k-resolution input images.
  arXiv  Detail & Related papers  (2020-04-01T13:52:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.