ReLoo: Reconstructing Humans Dressed in Loose Garments from Monocular Video in the Wild

• URL: http://arxiv.org/abs/2409.15269v2
• Date: Sat, 28 Sep 2024 22:24:56 GMT
• Title: ReLoo: Reconstructing Humans Dressed in Loose Garments from Monocular Video in the Wild
• Authors: Chen Guo, Tianjian Jiang, Manuel Kaufmann, Chengwei Zheng, Julien Valentin, Jie Song, Otmar Hilliges,
• Abstract summary: ReLoo reconstructs high-quality 3D models of humans dressed in loose garments from monocular in-the-wild videos. We first establish a layered neural human representation that decomposes clothed humans into a neural inner body and outer clothing. A global optimization jointly optimize the shape, appearance, and deformations of the human body and clothing via multi-layer differentiable volume rendering.
• Score: 33.7726643918619
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While previous years have seen great progress in the 3D reconstruction of humans from monocular videos, few of the state-of-the-art methods are able to handle loose garments that exhibit large non-rigid surface deformations during articulation. This limits the application of such methods to humans that are dressed in standard pants or T-shirts. Our method, ReLoo, overcomes this limitation and reconstructs high-quality 3D models of humans dressed in loose garments from monocular in-the-wild videos. To tackle this problem, we first establish a layered neural human representation that decomposes clothed humans into a neural inner body and outer clothing. On top of the layered neural representation, we further introduce a non-hierarchical virtual bone deformation module for the clothing layer that can freely move, which allows the accurate recovery of non-rigidly deforming loose clothing. A global optimization jointly optimizes the shape, appearance, and deformations of the human body and clothing via multi-layer differentiable volume rendering. To evaluate ReLoo, we record subjects with dynamically deforming garments in a multi-view capture studio. This evaluation, both on existing and our novel dataset, demonstrates ReLoo's clear superiority over prior art on both indoor datasets and in-the-wild videos.

Related papers

• DressRecon: Freeform 4D Human Reconstruction from Monocular Video [64.61230035671885]
  We present a method to reconstruct time-consistent human body models from monocular videos. We focus on extremely loose clothing or handheld object interactions. DressRecon yields higher-fidelity 3D reconstructions than prior art.
  arXiv  Detail & Related papers  (2024-09-30T17:59:15Z)
• TELA: Text to Layer-wise 3D Clothed Human Generation [27.93447899876341]
  This paper addresses the task of 3D clothed human generation from textural descriptions. We propose a layer-wise clothed human representation combined with a progressive optimization strategy. Our approach achieves state-of-the-art 3D clothed human generation while also supporting cloth editing applications.
  arXiv  Detail & Related papers  (2024-04-25T17:05:38Z)
• AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model [58.035758145894846]
  We introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos. A pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts. Our method is able to render natural garment dynamics that deviate highly from the body and well to generalize to both unseen views and poses.
  arXiv  Detail & Related papers  (2024-01-27T08:48:18Z)
• High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos [51.8323369577494]
  We propose the first method to recover high-quality animatable dynamic garments from monocular videos without depending on scanned data. To generate reasonable deformations for various unseen poses, we propose a learnable garment deformation network. We show that our method can reconstruct high-quality dynamic garments with coherent surface details, which can be easily animated under unseen poses.
  arXiv  Detail & Related papers  (2023-11-02T13:16:27Z)
• PERGAMO: Personalized 3D Garments from Monocular Video [6.8338761008826445]
  PERGAMO is a data-driven approach to learn a deformable model for 3D garments from monocular images. We first introduce a novel method to reconstruct the 3D geometry of garments from a single image, and use it to build a dataset of clothing from monocular videos. We show that our method is capable of producing garment animations that match the real-world behaviour, and generalizes to unseen body motions extracted from motion capture dataset.
  arXiv  Detail & Related papers  (2022-10-26T21:15:54Z)
• gDNA: Towards Generative Detailed Neural Avatars [94.9804106939663]
  We show that our model is able to generate natural human avatars wearing diverse and detailed clothing. Our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-01-11T18:46:38Z)
• High-Fidelity Neural Human Motion Transfer from Monocular Video [71.75576402562247]
  Video-based human motion transfer creates video animations of humans following a source motion. We present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations. In the experimental results, we significantly outperform the state-of-the-art in terms of video realism.
  arXiv  Detail & Related papers  (2020-12-20T16:54:38Z)
• MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video [10.679773937444445]
  We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. We build statistical deformation models for three types of clothing: T-shirt, short pants and long pants. Our method produces temporally coherent reconstruction of body and clothing from monocular video.
  arXiv  Detail & Related papers  (2020-09-22T17:54:38Z)
• Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images [50.34202789543989]
  Deep Fashion3D is the largest collection to date of 3D garment models. It provides rich annotations including 3D feature lines, 3D body pose and the corresponded multi-view real images. A novel adaptable template is proposed to enable the learning of all types of clothing in a single network.
  arXiv  Detail & Related papers  (2020-03-28T09:20:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.