3D Human Shape Reconstruction from a Polarization Image

• URL: http://arxiv.org/abs/2007.09268v1
• Date: Fri, 17 Jul 2020 22:36:02 GMT
• Title: 3D Human Shape Reconstruction from a Polarization Image
• Authors: Shihao Zou, Xinxin Zuo, Yiming Qian, Sen Wang, Chi Xu, Minglun Gong, Li Cheng
• Abstract summary: This paper tackles the problem of estimating 3D body shape of clothed humans from single polarized 2D images. A dedicated two-stage deep learning approach, SfP, is proposed.
• Score: 34.240256720930155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper tackles the problem of estimating 3D body shape of clothed humans from single polarized 2D images, i.e. polarization images. Polarization images are known to be able to capture polarized reflected lights that preserve rich geometric cues of an object, which has motivated its recent applications in reconstructing surface normal of the objects of interest. Inspired by the recent advances in human shape estimation from single color images, in this paper, we attempt at estimating human body shapes by leveraging the geometric cues from single polarization images. A dedicated two-stage deep learning approach, SfP, is proposed: given a polarization image, stage one aims at inferring the fined-detailed body surface normal; stage two gears to reconstruct the 3D body shape of clothing details. Empirical evaluations on a synthetic dataset (SURREAL) as well as a real-world dataset (PHSPD) demonstrate the qualitative and quantitative performance of our approach in estimating human poses and shapes. This indicates polarization camera is a promising alternative to the more conventional color or depth imaging for human shape estimation. Further, normal maps inferred from polarization imaging play a significant role in accurately recovering the body shapes of clothed people.

Related papers

• NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images [62.752710734332894]
  NeRSP is a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency. We achieve the state-of-the-art surface reconstruction results with only 6 views as input.
  arXiv  Detail & Related papers  (2024-06-11T09:53:18Z)
• S2P3: Self-Supervised Polarimetric Pose Prediction [55.43547228561919]
  This paper proposes the first self-supervised 6D object pose prediction from multimodal RGB+polarimetric images. The novel training paradigm comprises 1) a physical model to extract geometric information of polarized light, 2) a teacher-student knowledge distillation scheme and 3) a self-supervised loss formulation through differentiable constraints.
  arXiv  Detail & Related papers  (2023-12-02T10:46:40Z)
• Cloth2Body: Generating 3D Human Body Mesh from 2D Clothing [54.29207348918216]
  Cloth2Body needs to address new and emerging challenges raised by the partial observation of the input and the high diversity of the output. We propose an end-to-end framework that can accurately estimate 3D body mesh parameterized by pose and shape from a 2D clothing image. As shown by experimental results, the proposed framework achieves state-of-the-art performance and can effectively recover natural and diverse 3D body meshes from 2D images.
  arXiv  Detail & Related papers  (2023-09-28T06:18:38Z)
• Polarimetric Multi-View Inverse Rendering [13.391866136230165]
  A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) and the degree of polarization (DoP) of reflected light are related to an object's surface normal. We propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images.
  arXiv  Detail & Related papers  (2022-12-24T12:12:12Z)
• Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning [70.75369367311897]
  3D-aware global correspondences are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies. An adversarial generator takes the garment warped by the 3D-aware flow, and the image of the target person as inputs, to synthesize the photo-realistic try-on result.
  arXiv  Detail & Related papers  (2022-11-25T12:16:21Z)
• PolarFormer: Multi-camera 3D Object Detection with Polar Transformers [93.49713023975727]
  3D object detection in autonomous driving aims to reason "what" and "where" the objects of interest present in a 3D world. Existing methods often adopt the canonical Cartesian coordinate system with perpendicular axis. We propose a new Polar Transformer (PolarFormer) for more accurate 3D object detection in the bird's-eye-view (BEV) taking as input only multi-camera 2D images.
  arXiv  Detail & Related papers  (2022-06-30T16:32:48Z)
• Human Pose and Shape Estimation from Single Polarization Images [45.24275141578927]
  We attempt to estimate human pose and shape from single polarization images by leveraging the polarization-induced geometric cues. A dedicated dataset (PHSPD) is constructed, consisting of over 500K frames with accurate pose and shape annotations. It suggests polarization camera as a promising alternative to the more conventional RGB camera for human pose and shape estimation.
  arXiv  Detail & Related papers  (2021-08-15T22:56:18Z)
• Deep Polarization Imaging for 3D shape and SVBRDF Acquisition [7.86578678811226]
  We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints, we lift such restrictions by coupling polarization imaging with deep learning. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.
  arXiv  Detail & Related papers  (2021-05-06T17:58:43Z)
• Polarimetric Multi-View Inverse Rendering [13.391866136230165]
  A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) of reflected light is related to an object's surface normal. We propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that exploits geometric, photometric, and polarimetric cues extracted from input multi-view color polarization images.
  arXiv  Detail & Related papers  (2020-07-17T09:00:20Z)
• Polarization Human Shape and Pose Dataset [35.156049015251035]
  Polarization images are known to capture polarized reflected lights that preserve rich geometric cues of an object. Inspired by the recent breakthroughs in human shape estimation from a single color image, we attempt to investigate the new question of whether the geometric cues from polarization camera could be leveraged in estimating detailed human body shapes.
  arXiv  Detail & Related papers  (2020-04-30T15:58:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.