REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos
- URL: http://arxiv.org/abs/2305.14236v2
- Date: Sat, 27 May 2023 17:01:54 GMT
- Title: REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos
- Authors: Lingteng Qiu, Guanying Chen, Jiapeng Zhou, Mutian Xu, Junle Wang and
Xiaoguang Han
- Abstract summary: Reconstructing dynamic 3D garment surfaces with open boundaries from monocular videos is an important problem.
We introduce a novel approach, called REC-MV, to jointly optimize the explicit feature curves and the implicit signed distance field.
Our approach outperforms existing methods and can produce high-quality dynamic garment surfaces.
- Score: 23.25620556096607
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reconstructing dynamic 3D garment surfaces with open boundaries from
monocular videos is an important problem as it provides a practical and
low-cost solution for clothes digitization. Recent neural rendering methods
achieve high-quality dynamic clothed human reconstruction results from
monocular video, but these methods cannot separate the garment surface from the
body. Moreover, despite existing garment reconstruction methods based on
feature curve representation demonstrating impressive results for garment
reconstruction from a single image, they struggle to generate temporally
consistent surfaces for the video input. To address the above limitations, in
this paper, we formulate this task as an optimization problem of 3D garment
feature curves and surface reconstruction from monocular video. We introduce a
novel approach, called REC-MV, to jointly optimize the explicit feature curves
and the implicit signed distance field (SDF) of the garments. Then the open
garment meshes can be extracted via garment template registration in the
canonical space. Experiments on multiple casually captured datasets show that
our approach outperforms existing methods and can produce high-quality dynamic
garment surfaces. The source code is available at
https://github.com/GAP-LAB-CUHK-SZ/REC-MV.
Related papers
- Shape of Motion: 4D Reconstruction from a Single Video [51.04575075620677]
We introduce a method capable of reconstructing generic dynamic scenes, featuring explicit, full-sequence-long 3D motion.
We exploit the low-dimensional structure of 3D motion by representing scene motion with a compact set of SE3 motion bases.
Our method achieves state-of-the-art performance for both long-range 3D/2D motion estimation and novel view synthesis on dynamic scenes.
arXiv Detail & Related papers (2024-07-18T17:59:08Z) - SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix [60.48666051245761]
We propose a pose-free and training-free approach for generating 3D stereoscopic videos.
Our method warps a generated monocular video into camera views on stereoscopic baseline using estimated video depth.
We develop a disocclusion boundary re-injection scheme that further improves the quality of video inpainting.
arXiv Detail & Related papers (2024-06-29T08:33:55Z) - AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using
Garment Rigging Model [58.035758145894846]
We introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos.
A pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts.
Our method is able to render natural garment dynamics that deviate highly from the body and well to generalize to both unseen views and poses.
arXiv Detail & Related papers (2024-01-27T08:48:18Z) - DreaMo: Articulated 3D Reconstruction From A Single Casual Video [59.87221439498147]
We study articulated 3D shape reconstruction from a single and casually captured internet video, where the subject's view coverage is incomplete.
DreaMo shows promising quality in novel-view rendering, detailed articulated shape reconstruction, and skeleton generation.
arXiv Detail & Related papers (2023-12-05T09:47:37Z) - High-Quality Animatable Dynamic Garment Reconstruction from Monocular
Videos [51.8323369577494]
We propose the first method to recover high-quality animatable dynamic garments from monocular videos without depending on scanned data.
To generate reasonable deformations for various unseen poses, we propose a learnable garment deformation network.
We show that our method can reconstruct high-quality dynamic garments with coherent surface details, which can be easily animated under unseen poses.
arXiv Detail & Related papers (2023-11-02T13:16:27Z) - DiViNeT: 3D Reconstruction from Disparate Views via Neural Template
Regularization [7.488962492863031]
We present a volume rendering-based neural surface reconstruction method that takes as few as three disparate RGB images as input.
Our key idea is to regularize the reconstruction, which is severely ill-posed and leaving significant gaps between the sparse views.
Our approach achieves the best reconstruction quality among existing methods in the presence of such sparse views.
arXiv Detail & Related papers (2023-06-07T18:05:14Z) - PERGAMO: Personalized 3D Garments from Monocular Video [6.8338761008826445]
PERGAMO is a data-driven approach to learn a deformable model for 3D garments from monocular images.
We first introduce a novel method to reconstruct the 3D geometry of garments from a single image, and use it to build a dataset of clothing from monocular videos.
We show that our method is capable of producing garment animations that match the real-world behaviour, and generalizes to unseen body motions extracted from motion capture dataset.
arXiv Detail & Related papers (2022-10-26T21:15:54Z) - NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos [82.74918564737591]
We present a method for learning 3D geometry and physics parameters of a dynamic scene from only a monocular RGB video input.
Experiments show that our method achieves superior mesh and video reconstruction of dynamic scenes compared to competing Neural Field approaches.
arXiv Detail & Related papers (2022-10-22T04:57:55Z) - xCloth: Extracting Template-free Textured 3D Clothes from a Monocular
Image [4.056667956036515]
We present a novel framework for template-free textured 3D garment digitization.
More specifically, we propose to extend PeeledHuman representation to predict the pixel-aligned, layered depth and semantic maps.
We achieve high-fidelity 3D garment reconstruction results on three publicly available datasets and generalization on internet images.
arXiv Detail & Related papers (2022-08-27T05:57:00Z) - MonoClothCap: Towards Temporally Coherent Clothing Capture from
Monocular RGB Video [10.679773937444445]
We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input.
We build statistical deformation models for three types of clothing: T-shirt, short pants and long pants.
Our method produces temporally coherent reconstruction of body and clothing from monocular video.
arXiv Detail & Related papers (2020-09-22T17:54:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.