Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

• URL: http://arxiv.org/abs/2011.04965v1
• Date: Tue, 10 Nov 2020 08:14:36 GMT
• Title: Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion
• Authors: Yuhe Ding, Xin Ma, Mandi Luo, Aihua Zheng, Ran He
• Abstract summary: Photo-to-caricature aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. We propose an unsupervised contrastive photo-to-caricature translation architecture.
• Score: 49.93278173824292
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photo-to-caricature translation aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. To take both into consideration, we propose an unsupervised contrastive photo-to-caricature translation architecture. Considering the intuitive artifacts in the existing methods, we propose a contrastive style loss for style rendering to enforce the similarity between the style of rendered photo and the caricature, and simultaneously enhance its discrepancy to the photos. To obtain an exaggerating deformation in an unpaired/unsupervised fashion, we propose a Distortion Prediction Module (DPM) to predict a set of displacements vectors for each input image while fixing some controlling points, followed by the thin plate spline interpolation for warping. The model is trained on unpaired photo and caricature while can offer bidirectional synthesizing via inputting either a photo or a caricature. Extensive experiments demonstrate that the proposed model is effective to generate hand-drawn like caricatures compared with existing competitors.

Related papers

• Sketch-Guided Text-to-Image Diffusion Models [57.12095262189362]
  We introduce a universal approach to guide a pretrained text-to-image diffusion model. Our method does not require to train a dedicated model or a specialized encoder for the task. We take a particular focus on the sketch-to-image translation task, revealing a robust and expressive way to generate images.
  arXiv  Detail & Related papers  (2022-11-24T18:45:32Z)
• Unsupervised Scene Sketch to Photo Synthesis [40.044690369936184]
  We present a method for synthesizing realistic photos from scene sketches. Our framework learns from readily available large-scale photo datasets in an unsupervised manner. We also demonstrate that our framework facilitates a controllable manipulation of photo synthesis by editing strokes of corresponding sketches.
  arXiv  Detail & Related papers  (2022-09-06T22:25:06Z)
• Learning to Incorporate Texture Saliency Adaptive Attention to Image Cartoonization [20.578335938736384]
  A novel cartoon-texture-saliency-sampler (CTSS) module is proposed to dynamically sample cartoon-texture-salient patches from training data. With extensive experiments, we demonstrate that texture saliency adaptive attention in adversarial learning, is of significant importance in facilitating and enhancing image cartoonization.
  arXiv  Detail & Related papers  (2022-08-02T16:45:55Z)
• Unsupervised Coherent Video Cartoonization with Perceptual Motion Consistency [89.75731026852338]
  We propose a spatially-adaptive alignment framework with perceptual motion consistency for coherent video cartoonization. We devise the semantic correlative map as a style-independent, global-aware regularization on the perceptual consistency motion. Our method is able to generate highly stylistic and temporal consistent cartoon videos.
  arXiv  Detail & Related papers  (2022-04-02T07:59:02Z)
• CariMe: Unpaired Caricature Generation with Multiple Exaggerations [22.342630945133312]
  Caricature generation aims to translate real photos into caricatures with artistic styles and shape exaggerations. Previous caricature generation methods are obsessed with predicting definite image warping from a given photo. We propose a Multi-exaggeration Warper network to learn the distribution-level mapping from photo to facial exaggerations.
  arXiv  Detail & Related papers  (2020-10-01T08:14:32Z)
• DeepFacePencil: Creating Face Images from Freehand Sketches [77.00929179469559]
  Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision. We propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches.
  arXiv  Detail & Related papers  (2020-08-31T03:35:21Z)
• Learning to Caricature via Semantic Shape Transform [95.25116681761142]
  We propose an algorithm based on a semantic shape transform to produce shape exaggerations. We show that the proposed framework is able to render visually pleasing shape exaggerations while maintaining their facial structures.
  arXiv  Detail & Related papers  (2020-08-12T03:41:49Z)
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches [133.01690754567252]
  Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches. Deep Plastic Surgery is a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs.
  arXiv  Detail & Related papers  (2020-01-09T08:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.