3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos

• URL: http://arxiv.org/abs/2003.06841v2
• Date: Sat, 13 Nov 2021 01:27:47 GMT
• Title: 3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos
• Authors: Zipeng Ye, Mengfei Xia, Yanan Sun, Ran Yi, Minjing Yu, Juyong Zhang, Yu-Kun Lai, Yong-jin Liu
• Abstract summary: We propose an end-to-end deep neural network model that generates high-quality 3D caricatures directly from a normal 2D face photo. Experiments including a novel two-level user study show that our system can generate high-quality 3D caricatures directly from normal face photos.
• Score: 78.14395302760148
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Caricature is a type of artistic style of human faces that attracts considerable attention in the entertainment industry. So far a few 3D caricature generation methods exist and all of them require some caricature information (e.g., a caricature sketch or 2D caricature) as input. This kind of input, however, is difficult to provide by non-professional users. In this paper, we propose an end-to-end deep neural network model that generates high-quality 3D caricatures directly from a normal 2D face photo. The most challenging issue for our system is that the source domain of face photos (characterized by normal 2D faces) is significantly different from the target domain of 3D caricatures (characterized by 3D exaggerated face shapes and textures). To address this challenge, we: (1) build a large dataset of 5,343 3D caricature meshes and use it to establish a PCA model in the 3D caricature shape space; (2) reconstruct a normal full 3D head from the input face photo and use its PCA representation in the 3D caricature shape space to establish correspondences between the input photo and 3D caricature shape; and (3) propose a novel character loss and a novel caricature loss based on previous psychological studies on caricatures. Experiments including a novel two-level user study show that our system can generate high-quality 3D caricatures directly from normal face photos.

Related papers

• Toon3D: Seeing Cartoons from a New Perspective [52.85312338932685]
  We focus our analysis on hand-drawn images from cartoons and anime. Many cartoons are created by artists without a 3D rendering engine, which means that any new image of a scene is hand-drawn. We correct for 2D drawing inconsistencies to recover a plausible 3D structure such that the newly warped drawings are consistent with each other.
  arXiv  Detail & Related papers  (2024-05-16T17:59:51Z)
• High-Quality Face Caricature via Style Translation [1.3457834965263997]
  We propose a high-quality, unpaired face caricature method that is appropriate for use in the real world. We attain the exaggeration of facial features and the stylization of appearance through a two-step process. The Face caricature projection employs an encoder trained with real and caricature faces with the pretrained generator to project real and caricature faces.
  arXiv  Detail & Related papers  (2023-11-22T12:03:33Z)
• Generating Animatable 3D Cartoon Faces from Single Portraits [51.15618892675337]
  We present a novel framework to generate animatable 3D cartoon faces from a single portrait image. We propose a two-stage reconstruction method to recover the 3D cartoon face with detailed texture. Finally, we propose a semantic preserving face rigging method based on manually created templates and deformation transfer.
  arXiv  Detail & Related papers  (2023-07-04T04:12:50Z)
• Deep Deformable 3D Caricatures with Learned Shape Control [25.5491131982863]
  A 3D caricature is an exaggerated 3D depiction of a human face. In this paper, we propose a caricature-based framework for building a deformable surface model. We create variations of 3D surfaces by learning by learning the parameters of a deformable model.
  arXiv  Detail & Related papers  (2022-07-29T10:21:27Z)
• 3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image [142.047662926209]
  We generate 3D cartoon face shapes from single 2D GAN generated human faces. We manipulate latent codes to generate images with different poses and lighting, such that we can reconstruct the 3D cartoon face shapes.
  arXiv  Detail & Related papers  (2022-07-29T01:06:21Z)
• 3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction [23.539931080533226]
  3DCaricShop is the first large-scale 3D caricature dataset that contains 2000 high-quality diversified 3D caricatures manually crafted by professional artists. 3DCaricShop also provides rich annotations including a paired 2D caricature image, camera parameters and 3D facial landmarks. We propose a novel view-collaborative graph convolution network (VCGCN) to extract key points from the implicit mesh for accurate alignment.
  arXiv  Detail & Related papers  (2021-03-15T08:24:29Z)
• FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction [62.5557724039217]
  Facial expressions induce a variety of high-level details on the 3D face geometry. Morphable Models (3DMMs) of the human face fail to capture such fine details in their PCA-based representations. We introduce FaceDet3D, a first-of-its-kind method that generates - from a single image - geometric facial details consistent with any desired target expression.
  arXiv  Detail & Related papers  (2020-12-14T23:07:38Z)
• Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture [58.78290175562601]
  This paper presents a solution to the problem of deforming an artist-drawn caricature according to a given normal face expression. The key of our solution is a novel method to model caricature expression, which extends traditional 3DMM representation to caricature domain. The experiments demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2020-08-13T06:31:01Z)
• Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model [27.553158595012974]
  We propose the first automatic method for automatic landmark detection and 3D face reconstruction for caricature. Based on the constructed dataset and the nonlinear parametric model, we propose a neural network based method to regress the 3D face shape and orientation from the input 2D caricature image.
  arXiv  Detail & Related papers  (2020-04-20T10:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.