Learning to Incorporate Texture Saliency Adaptive Attention to Image Cartoonization

• URL: http://arxiv.org/abs/2208.01587v4
• Date: Sun, 18 Feb 2024 12:13:12 GMT
• Title: Learning to Incorporate Texture Saliency Adaptive Attention to Image Cartoonization
• Authors: Xiang Gao, Yuqi Zhang, and Yingjie Tian
• Abstract summary: A novel cartoon-texture-saliency-sampler (CTSS) module is proposed to dynamically sample cartoon-texture-salient patches from training data. With extensive experiments, we demonstrate that texture saliency adaptive attention in adversarial learning, is of significant importance in facilitating and enhancing image cartoonization.
• Score: 20.578335938736384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image cartoonization is recently dominated by generative adversarial networks (GANs) from the perspective of unsupervised image-to-image translation, in which an inherent challenge is to precisely capture and sufficiently transfer characteristic cartoon styles (e.g., clear edges, smooth color shading, abstract fine structures, etc.). Existing advanced models try to enhance cartoonization effect by learning to promote edges adversarially, introducing style transfer loss, or learning to align style from multiple representation space. This paper demonstrates that more distinct and vivid cartoonization effect could be easily achieved with only basic adversarial loss. Observing that cartoon style is more evident in cartoon-texture-salient local image regions, we build a region-level adversarial learning branch in parallel with the normal image-level one, which constrains adversarial learning on cartoon-texture-salient local patches for better perceiving and transferring cartoon texture features. To this end, a novel cartoon-texture-saliency-sampler (CTSS) module is proposed to dynamically sample cartoon-texture-salient patches from training data. With extensive experiments, we demonstrate that texture saliency adaptive attention in adversarial learning, as a missing ingredient of related methods in image cartoonization, is of significant importance in facilitating and enhancing image cartoon stylization, especially for high-resolution input pictures.

Related papers

• Instance-guided Cartoon Editing with a Large-scale Dataset [12.955181769243232]
  We present an instance-aware image segmentation model that can generate accurate, high-resolution segmentation masks for characters in cartoon images. We present that the proposed approach enables a range of segmentation-dependent cartoon editing applications like 3D Ken Burns parallax effects, text-guided cartoon style editing, and puppet animation from illustrations and manga.
  arXiv  Detail & Related papers  (2023-12-04T15:00:15Z)
• Portrait Diffusion: Training-free Face Stylization with Chain-of-Painting [64.43760427752532]
  Face stylization refers to the transformation of a face into a specific portrait style. Current methods require the use of example-based adaptation approaches to fine-tune pre-trained generative models. This paper proposes a training-free face stylization framework, named Portrait Diffusion.
  arXiv  Detail & Related papers  (2023-12-03T06:48:35Z)
• Interactive Cartoonization with Controllable Perceptual Factors [5.8641445422054765]
  We propose a novel solution with editing features of texture and color based on the cartoon creation process. In the texture decoder, we propose a texture controller, which enables a user to control stroke style and abstraction to generate diverse cartoon textures. We also introduce an HSV color augmentation to induce the networks to generate diverse and controllable color translation.
  arXiv  Detail & Related papers  (2022-12-19T15:45:47Z)
• Unsupervised Coherent Video Cartoonization with Perceptual Motion Consistency [89.75731026852338]
  We propose a spatially-adaptive alignment framework with perceptual motion consistency for coherent video cartoonization. We devise the semantic correlative map as a style-independent, global-aware regularization on the perceptual consistency motion. Our method is able to generate highly stylistic and temporal consistent cartoon videos.
  arXiv  Detail & Related papers  (2022-04-02T07:59:02Z)
• White-Box Cartoonization Using An Extended GAN Framework [0.0]
  We propose to implement a new framework for estimating generative models via an adversarial process to extend an existing GAN framework. We develop a white-box controllable image cartoonization, which can generate high-quality cartooned images/videos from real-world photos and videos.
  arXiv  Detail & Related papers  (2021-07-09T17:09:19Z)
• MarioNette: Self-Supervised Sprite Learning [67.51317291061115]
  We propose a deep learning approach for obtaining a graphically disentangled representation of recurring elements. By jointly learning a dictionary of texture patches and training a network that places them onto a canvas, we effectively deconstruct sprite-based content into a sparse, consistent, and interpretable representation.
  arXiv  Detail & Related papers  (2021-04-29T17:59:01Z)
• Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion [49.93278173824292]
  Photo-to-caricature aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. We propose an unsupervised contrastive photo-to-caricature translation architecture.
  arXiv  Detail & Related papers  (2020-11-10T08:14:36Z)
• Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture [58.78290175562601]
  This paper presents a solution to the problem of deforming an artist-drawn caricature according to a given normal face expression. The key of our solution is a novel method to model caricature expression, which extends traditional 3DMM representation to caricature domain. The experiments demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2020-08-13T06:31:01Z)
• Learning to Caricature via Semantic Shape Transform [95.25116681761142]
  We propose an algorithm based on a semantic shape transform to produce shape exaggerations. We show that the proposed framework is able to render visually pleasing shape exaggerations while maintaining their facial structures.
  arXiv  Detail & Related papers  (2020-08-12T03:41:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.