High-Fidelity Clothed Avatar Reconstruction from a Single Image

• URL: http://arxiv.org/abs/2304.03903v1
• Date: Sat, 8 Apr 2023 04:01:04 GMT
• Title: High-Fidelity Clothed Avatar Reconstruction from a Single Image
• Authors: Tingting Liao and Xiaomei Zhang and Yuliang Xiu and Hongwei Yi and Xudong Liu and Guo-Jun Qi and Yong Zhang and Xuan Wang and Xiangyu Zhu and Zhen Lei
• Abstract summary: We propose a coarse-to-fine way to realize a high-fidelity clothed avatar reconstruction from a single image. We use an implicit model to learn the general shape in the canonical space of a person in a learning-based way. We refine the surface detail by estimating the non-rigid deformation in the posed space in an optimization way.
• Score: 73.15939963381906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a framework for efficient 3D clothed avatar reconstruction. By combining the advantages of the high accuracy of optimization-based methods and the efficiency of learning-based methods, we propose a coarse-to-fine way to realize a high-fidelity clothed avatar reconstruction (CAR) from a single image. At the first stage, we use an implicit model to learn the general shape in the canonical space of a person in a learning-based way, and at the second stage, we refine the surface detail by estimating the non-rigid deformation in the posed space in an optimization way. A hyper-network is utilized to generate a good initialization so that the convergence o f the optimization process is greatly accelerated. Extensive experiments on various datasets show that the proposed CAR successfully produces high-fidelity avatars for arbitrarily clothed humans in real scenes.

Related papers

• OrientDream: Streamlining Text-to-3D Generation with Explicit Orientation Control [66.03885917320189]
  OrientDream is a camera orientation conditioned framework for efficient and multi-view consistent 3D generation from textual prompts. Our strategy emphasizes the implementation of an explicit camera orientation conditioned feature in the pre-training of a 2D text-to-image diffusion module. Our experiments reveal that our method not only produces high-quality NeRF models with consistent multi-view properties but also achieves an optimization speed significantly greater than existing methods.
  arXiv  Detail & Related papers  (2024-06-14T13:16:18Z)
• Adaptive Image Registration: A Hybrid Approach Integrating Deep Learning and Optimization Functions for Enhanced Precision [13.242184146186974]
  We propose a single framework for image registration based on deep neural networks and optimization. We show improvements of up to 1.6% in test data, while maintaining the same inference time, and a substantial 1.0% points performance gain in deformation field smoothness.
  arXiv  Detail & Related papers  (2023-11-27T02:48:06Z)
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion [105.16622018766236]
  Wonder3D is a novel method for efficiently generating high-fidelity textured meshes from single-view images. To holistically improve the quality, consistency, and efficiency of image-to-3D tasks, we propose a cross-domain diffusion model.
  arXiv  Detail & Related papers  (2023-10-23T15:02:23Z)
• Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable Rendering Method [19.330797817738542]
  We introduce a novel hybrid differentiable rendering method to efficiently reconstruct the 3D geometry and reflectance of a scene. Our method can produce reconstructions with similar or higher quality than state-of-the-art methods while being more efficient.
  arXiv  Detail & Related papers  (2023-08-19T12:48:10Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• Inverting Generative Adversarial Renderer for Face Reconstruction [58.45125455811038]
  In this work, we introduce a novel Generative Adversa Renderer (GAR) GAR learns to model the complicated real-world image, instead of relying on the graphics rules, it is capable of producing realistic images. Our method achieves state-of-the-art performances on multiple face reconstruction.
  arXiv  Detail & Related papers  (2021-05-06T04:16:06Z)
• Riggable 3D Face Reconstruction via In-Network Optimization [58.016067611038046]
  This paper presents a method for riggable 3D face reconstruction from monocular images. It jointly estimates a personalized face rig and per-image parameters including expressions, poses, and illuminations. Experiments demonstrate that our method achieves SOTA reconstruction accuracy, reasonable robustness and generalization ability.
  arXiv  Detail & Related papers  (2021-04-08T03:53:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.