Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture

• URL: http://arxiv.org/abs/2008.05714v1
• Date: Thu, 13 Aug 2020 06:31:01 GMT
• Title: Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture
• Authors: Keyu Chen, Jianmin Zheng, Jianfei Cai, Juyong Zhang
• Abstract summary: This paper presents a solution to the problem of deforming an artist-drawn caricature according to a given normal face expression. The key of our solution is a novel method to model caricature expression, which extends traditional 3DMM representation to caricature domain. The experiments demonstrate the effectiveness of the proposed method.
• Score: 58.78290175562601
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The problem of deforming an artist-drawn caricature according to a given normal face expression is of interest in applications such as social media, animation and entertainment. This paper presents a solution to the problem, with an emphasis on enhancing the ability to create desired expressions and meanwhile preserve the identity exaggeration style of the caricature, which imposes challenges due to the complicated nature of caricatures. The key of our solution is a novel method to model caricature expression, which extends traditional 3DMM representation to caricature domain. The method consists of shape modelling and texture generation for caricatures. Geometric optimization is developed to create identity-preserving blendshapes for reconstructing accurate and stable geometric shape, and a conditional generative adversarial network (cGAN) is designed for generating dynamic textures under target expressions. The combination of both shape and texture components makes the non-trivial expressions of a caricature be effectively defined by the extension of the popular 3DMM representation and a caricature can thus be flexibly deformed into arbitrary expressions with good results visually in both shape and color spaces. The experiments demonstrate the effectiveness of the proposed method.

Related papers

• High-Quality Face Caricature via Style Translation [1.3457834965263997]
  We propose a high-quality, unpaired face caricature method that is appropriate for use in the real world. We attain the exaggeration of facial features and the stylization of appearance through a two-step process. The Face caricature projection employs an encoder trained with real and caricature faces with the pretrained generator to project real and caricature faces.
  arXiv  Detail & Related papers  (2023-11-22T12:03:33Z)
• TADA! Text to Animatable Digital Avatars [57.52707683788961]
  TADA takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures. We derive an optimizable high-resolution body model from SMPL-X with 3D displacements and a texture map. We render normals and RGB images of the generated character and exploit their latent embeddings in the SDS training process.
  arXiv  Detail & Related papers  (2023-08-21T17:59:10Z)
• Generating Animatable 3D Cartoon Faces from Single Portraits [51.15618892675337]
  We present a novel framework to generate animatable 3D cartoon faces from a single portrait image. We propose a two-stage reconstruction method to recover the 3D cartoon face with detailed texture. Finally, we propose a semantic preserving face rigging method based on manually created templates and deformation transfer.
  arXiv  Detail & Related papers  (2023-07-04T04:12:50Z)
• Neural Strokes: Stylized Line Drawing of 3D Shapes [36.88356061690497]
  This paper introduces a model for producing stylized line drawings from 3D shapes. The model takes a 3D shape and a viewpoint as input, and outputs a drawing with textured strokes.
  arXiv  Detail & Related papers  (2021-10-08T05:40:57Z)
• Learning to Caricature via Semantic Shape Transform [95.25116681761142]
  We propose an algorithm based on a semantic shape transform to produce shape exaggerations. We show that the proposed framework is able to render visually pleasing shape exaggerations while maintaining their facial structures.
  arXiv  Detail & Related papers  (2020-08-12T03:41:49Z)
• Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model [27.553158595012974]
  We propose the first automatic method for automatic landmark detection and 3D face reconstruction for caricature. Based on the constructed dataset and the nonlinear parametric model, we propose a neural network based method to regress the 3D face shape and orientation from the input 2D caricature image.
  arXiv  Detail & Related papers  (2020-04-20T10:34:52Z)
• 3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos [78.14395302760148]
  We propose an end-to-end deep neural network model that generates high-quality 3D caricatures directly from a normal 2D face photo. Experiments including a novel two-level user study show that our system can generate high-quality 3D caricatures directly from normal face photos.
  arXiv  Detail & Related papers  (2020-03-15T14:42:15Z)
• MW-GAN: Multi-Warping GAN for Caricature Generation with Multi-Style Geometric Exaggeration [53.98437317161086]
  Given an input face photo, the goal of caricature generation is to produce stylized, exaggerated caricatures that share the same identity as the photo. We propose a novel framework called Multi-Warping GAN (MW-GAN), including a style network and a geometric network. Experiments show that caricatures generated by MW-GAN have better quality than existing methods.
  arXiv  Detail & Related papers  (2020-01-07T03:08:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.