BodyMap: Learning Full-Body Dense Correspondence Map
- URL: http://arxiv.org/abs/2205.09111v1
- Date: Wed, 18 May 2022 17:58:11 GMT
- Title: BodyMap: Learning Full-Body Dense Correspondence Map
- Authors: Anastasia Ianina, Nikolaos Sarafianos, Yuanlu Xu, Ignacio Rocco, Tony
Tung
- Abstract summary: BodyMap is a new framework for obtaining high-definition full-body and continuous dense correspondence between in-the-wild images of humans and the surface of a 3D template model.
Dense correspondence between humans carries powerful semantic information that can be utilized to solve fundamental problems for full-body understanding.
- Score: 19.13654133912062
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Dense correspondence between humans carries powerful semantic information
that can be utilized to solve fundamental problems for full-body understanding
such as in-the-wild surface matching, tracking and reconstruction. In this
paper we present BodyMap, a new framework for obtaining high-definition
full-body and continuous dense correspondence between in-the-wild images of
clothed humans and the surface of a 3D template model. The correspondences
cover fine details such as hands and hair, while capturing regions far from the
body surface, such as loose clothing. Prior methods for estimating such dense
surface correspondence i) cut a 3D body into parts which are unwrapped to a 2D
UV space, producing discontinuities along part seams, or ii) use a single
surface for representing the whole body, but none handled body details. Here,
we introduce a novel network architecture with Vision Transformers that learn
fine-level features on a continuous body surface. BodyMap outperforms prior
work on various metrics and datasets, including DensePose-COCO by a large
margin. Furthermore, we show various applications ranging from multi-layer
dense cloth correspondence, neural rendering with novel-view synthesis and
appearance swapping.
Related papers
- MOSE: Monocular Semantic Reconstruction Using NeRF-Lifted Noisy Priors [11.118490283303407]
We propose a neural field semantic reconstruction approach to lift inferred image-level noisy priors to 3D.
Our method produces accurate semantics and geometry in both 3D and 2D space.
arXiv Detail & Related papers (2024-09-21T05:12:13Z) - 3D Reconstruction of Interacting Multi-Person in Clothing from a Single Image [8.900009931200955]
This paper introduces a novel pipeline to reconstruct the geometry of interacting multi-person in clothing on a globally coherent scene space from a single image.
We overcome this challenge by utilizing two human priors for complete 3D geometry and surface contacts.
The results demonstrate that our method is complete, globally coherent, and physically plausible compared to existing methods.
arXiv Detail & Related papers (2024-01-12T07:23:02Z) - Exploring Shape Embedding for Cloth-Changing Person Re-Identification
via 2D-3D Correspondences [9.487097819140653]
We propose a new shape embedding paradigm for cloth-changing ReID.
The shape embedding paradigm based on 2D-3D correspondences remarkably enhances the model's global understanding of human body shape.
To promote the study of ReID under clothing change, we construct 3D Dense Persons (DP3D), which is the first large-scale cloth-changing ReID dataset.
arXiv Detail & Related papers (2023-10-27T19:26:30Z) - DECO: Dense Estimation of 3D Human-Scene Contact In The Wild [54.44345845842109]
We train a novel 3D contact detector that uses both body-part-driven and scene-context-driven attention to estimate contact on the SMPL body.
We significantly outperform existing SOTA methods across all benchmarks.
We also show qualitatively that DECO generalizes well to diverse and challenging real-world human interactions in natural images.
arXiv Detail & Related papers (2023-09-26T21:21:07Z) - Generalizable Neural Performer: Learning Robust Radiance Fields for
Human Novel View Synthesis [52.720314035084215]
This work targets at using a general deep learning framework to synthesize free-viewpoint images of arbitrary human performers.
We present a simple yet powerful framework, named Generalizable Neural Performer (GNR), that learns a generalizable and robust neural body representation.
Experiments on GeneBody-1.0 and ZJU-Mocap show better robustness of our methods than recent state-of-the-art generalizable methods.
arXiv Detail & Related papers (2022-04-25T17:14:22Z) - DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to
the Third Dimension [71.71234436165255]
We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only.
Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species.
We show significant improvements compared to state-of-the-art non-rigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals.
arXiv Detail & Related papers (2021-08-31T18:33:55Z) - Detailed Avatar Recovery from Single Image [50.82102098057822]
This paper presents a novel framework to recover emphdetailed avatar from a single image.
We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework.
Our method can restore detailed human body shapes with complete textures beyond skinned models.
arXiv Detail & Related papers (2021-08-06T03:51:26Z) - HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences [60.89437526374286]
Prior art either assumes small motion between frames or relies on local descriptors, which cannot handle large motion or visually ambiguous body parts.
We propose a deep learning framework that maps each pixel to a feature space, where the feature distances reflect the geodesic distances among pixels.
Without any semantic annotation, the proposed embeddings automatically learn to differentiate visually similar parts and align different subjects into an unified feature space.
arXiv Detail & Related papers (2021-03-29T12:43:44Z) - Combining Implicit Function Learning and Parametric Models for 3D Human
Reconstruction [123.62341095156611]
Implicit functions represented as deep learning approximations are powerful for reconstructing 3D surfaces.
Such features are essential in building flexible models for both computer graphics and computer vision.
We present methodology that combines detail-rich implicit functions and parametric representations.
arXiv Detail & Related papers (2020-07-22T13:46:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.