3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction

• URL: http://arxiv.org/abs/2103.08204v1
• Date: Mon, 15 Mar 2021 08:24:29 GMT
• Title: 3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction
• Authors: Yuda Qiu, Xiaojie Xu, Lingteng Qiu, Yan Pan, Yushuang Wu, Weikai Chen, Xiaoguang Han
• Abstract summary: 3DCaricShop is the first large-scale 3D caricature dataset that contains 2000 high-quality diversified 3D caricatures manually crafted by professional artists. 3DCaricShop also provides rich annotations including a paired 2D caricature image, camera parameters and 3D facial landmarks. We propose a novel view-collaborative graph convolution network (VCGCN) to extract key points from the implicit mesh for accurate alignment.
• Score: 23.539931080533226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Caricature is an artistic representation that deliberately exaggerates the distinctive features of a human face to convey humor or sarcasm. However, reconstructing a 3D caricature from a 2D caricature image remains a challenging task, mostly due to the lack of data. We propose to fill this gap by introducing 3DCaricShop, the first large-scale 3D caricature dataset that contains 2000 high-quality diversified 3D caricatures manually crafted by professional artists. 3DCaricShop also provides rich annotations including a paired 2D caricature image, camera parameters and 3D facial landmarks. To demonstrate the advantage of 3DCaricShop, we present a novel baseline approach for single-view 3D caricature reconstruction. To ensure a faithful reconstruction with plausible face deformations, we propose to connect the good ends of the detailrich implicit functions and the parametric mesh representations. In particular, we first register a template mesh to the output of the implicit generator and iteratively project the registration result onto a pre-trained PCA space to resolve artifacts and self-intersections. To deal with the large deformation during non-rigid registration, we propose a novel view-collaborative graph convolution network (VCGCN) to extract key points from the implicit mesh for accurate alignment. Our method is able to generate highfidelity 3D caricature in a pre-defined mesh topology that is animation-ready. Extensive experiments have been conducted on 3DCaricShop to verify the significance of the database and the effectiveness of the proposed method.

Related papers

• FAMOUS: High-Fidelity Monocular 3D Human Digitization Using View Synthesis [51.193297565630886]
  The challenge of accurately inferring texture remains, particularly in obscured areas such as the back of a person in frontal-view images. This limitation in texture prediction largely stems from the scarcity of large-scale and diverse 3D datasets. We propose leveraging extensive 2D fashion datasets to enhance both texture and shape prediction in 3D human digitization.
  arXiv  Detail & Related papers  (2024-10-13T01:25:05Z)
• ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling [96.87575334960258]
  ID-to-3D is a method to generate identity- and text-guided 3D human heads with disentangled expressions. Results achieve an unprecedented level of identity-consistent and high-quality texture and geometry generation.
  arXiv  Detail & Related papers  (2024-05-26T13:36:45Z)
• LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image [64.94932577552458]
  Large Reconstruction Models have made significant strides in the realm of automated 3D content generation from single or multiple input images. Despite their success, these models often produce 3D meshes with geometric inaccuracies, stemming from the inherent challenges of deducing 3D shapes solely from image data. We introduce a novel framework, the Large Image and Point Cloud Alignment Model (LAM3D), which utilizes 3D point cloud data to enhance the fidelity of generated 3D meshes.
  arXiv  Detail & Related papers  (2024-05-24T15:09:12Z)
• Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation [55.73399465968594]
  This paper proposes a novel generation paradigm Sketch3D to generate realistic 3D assets with shape aligned with the input sketch and color matching the textual description. Three strategies are designed to optimize 3D Gaussians, i.e., structural optimization via a distribution transfer mechanism, color optimization with a straightforward MSE loss and sketch similarity optimization with a CLIP-based geometric similarity loss.
  arXiv  Detail & Related papers  (2024-04-02T11:03:24Z)
• Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture [47.44029968307207]
  We propose a novel framework for simultaneous high-fidelity recovery of object shapes and textures from single-view images. Our approach utilizes the proposed Single-view neural implicit Shape and Radiance field (SSR) representations to leverage both explicit 3D shape supervision and volume rendering. A distinctive feature of our framework is its ability to generate fine-grained textured meshes while seamlessly integrating rendering capabilities into the single-view 3D reconstruction model.
  arXiv  Detail & Related papers  (2023-11-01T11:46:15Z)
• RaBit: Parametric Modeling of 3D Biped Cartoon Characters with a Topological-consistent Dataset [19.494054103293116]
  We introduce 3DBiCar, the first large-scale dataset of 3D biped cartoon characters, and RaBit, the corresponding parametric model. RaBit is built with a SMPL-like linear blend shape model and a StyleGAN-based neural UV-texture generator, simultaneously expressing the shape, pose, and texture. Various applications are conducted, including single-view reconstruction, sketch-based modeling, and 3D cartoon animation.
  arXiv  Detail & Related papers  (2023-03-22T13:46:15Z)
• SSR-2D: Semantic 3D Scene Reconstruction from 2D Images [54.46126685716471]
  In this work, we explore a central 3D scene modeling task, namely, semantic scene reconstruction without using any 3D annotations. The key idea of our approach is to design a trainable model that employs both incomplete 3D reconstructions and their corresponding source RGB-D images. Our method achieves the state-of-the-art performance of semantic scene completion on two large-scale benchmark datasets MatterPort3D and ScanNet.
  arXiv  Detail & Related papers  (2023-02-07T17:47:52Z)
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape [77.95154911528365]
  3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. Previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry. This paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person.
  arXiv  Detail & Related papers  (2022-04-09T03:46:18Z)
• Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model [27.553158595012974]
  We propose the first automatic method for automatic landmark detection and 3D face reconstruction for caricature. Based on the constructed dataset and the nonlinear parametric model, we propose a neural network based method to regress the 3D face shape and orientation from the input 2D caricature image.
  arXiv  Detail & Related papers  (2020-04-20T10:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.