MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing

• URL: http://arxiv.org/abs/2004.10634v2
• Date: Thu, 17 Dec 2020 17:21:42 GMT
• Title: MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing
• Authors: Hao Su, Jianwei Niu, Xuefeng Liu, Qingfeng Li, Jiahe Cui, and Ji Wan
• Abstract summary: We propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation. Inspired by how experienced manga artists draw manga, MangaGAN generates the geometric features of manga face by a designed GAN model. To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module.
• Score: 27.99490750445691
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Manga is a world popular comic form originated in Japan, which typically employs black-and-white stroke lines and geometric exaggeration to describe humans' appearances, poses, and actions. In this paper, we propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation. Inspired by how experienced manga artists draw manga, MangaGAN generates the geometric features of manga face by a designed GAN model and delicately translates each facial region into the manga domain by a tailored multi-GANs architecture. For training MangaGAN, we construct a new dataset collected from a popular manga work, containing manga facial features, landmarks, bodies, and so on. Moreover, to produce high-quality manga faces, we further propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module to improve the similarity between domains of photo and manga. Extensive experiments show that MangaGAN can produce high-quality manga faces which preserve both the facial similarity and a popular manga style, and outperforms other related state-of-the-art methods.

Related papers

• MangaUB: A Manga Understanding Benchmark for Large Multimodal Models [25.63892470012361]
  Manga is a popular medium that combines stylized drawings and text to convey stories. Recently, the adaptive nature of modern large multimodal models (LMMs) shows possibilities for more general approaches. MangaUB is designed to assess the recognition and understanding of content shown in a single panel as well as conveyed across multiple panels.
  arXiv  Detail & Related papers  (2024-07-26T18:21:30Z)
• Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models [26.010509997863196]
  We propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga. Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.
  arXiv  Detail & Related papers  (2024-03-13T05:33:52Z)
• The Manga Whisperer: Automatically Generating Transcriptions for Comics [55.544015596503726]
  We present a unified model, Magi, that is able to detect panels, text boxes and character boxes. We propose a novel approach that is able to sort the detected text boxes in their reading order and generate a dialogue transcript.
  arXiv  Detail & Related papers  (2024-01-18T18:59:09Z)
• inkn'hue: Enhancing Manga Colorization from Multiple Priors with Alignment Multi-Encoder VAE [0.0]
  We propose a specialized framework for manga colorization. We leverage established models for shading and vibrant coloring using a multi-encoder VAE. This structured workflow ensures clear and colorful results, with the option to incorporate reference images and manual hints.
  arXiv  Detail & Related papers  (2023-11-03T09:33:32Z)
• DeepPortraitDrawing: Generating Human Body Images from Freehand Sketches [75.4318318890065]
  We present DeepDrawing, a framework for converting roughly drawn sketches to realistic human body images. To encode complicated body shapes under various poses, we take a local-to-global approach. Our method produces more realistic images than the state-of-the-art sketch-to-image synthesis techniques.
  arXiv  Detail & Related papers  (2022-05-04T14:02:45Z)
• Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering [21.696847342192072]
  The artistic expression and stylistic limitations of manga pose many challenges to the re-identification problem. Inspired by the idea that some content-related features may help clustering, we propose a Face-body and Spatial-temporal Associated Clustering method. In the face-body combination module, a face-body graph is constructed to solve problems such as exaggeration and deformation in artistic creation. In the spatial-temporal relationship correction module, we analyze the appearance features of characters and design a temporal-spatial-related triplet loss to fine-tune the clustering.
  arXiv  Detail & Related papers  (2022-04-10T07:28:41Z)
• HumanGAN: A Generative Model of Humans Images [78.6284090004218]
  We present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style. Our model encodes part-based latent appearance vectors in a normalized pose-independent space and warps them to different poses, it preserves body and clothing appearance under varying posture.
  arXiv  Detail & Related papers  (2021-03-11T19:00:38Z)
• AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation [84.52819242283852]
  We propose a novel framework to translate a portrait photo-face into an anime appearance. Our aim is to synthesize anime-faces which are style-consistent with a given reference anime-face. Existing methods often fail to transfer the styles of reference anime-faces, or introduce noticeable artifacts/distortions in the local shapes of their generated faces.
  arXiv  Detail & Related papers  (2021-02-24T22:47:38Z)
• DeepFacePencil: Creating Face Images from Freehand Sketches [77.00929179469559]
  Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision. We propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches.
  arXiv  Detail & Related papers  (2020-08-31T03:35:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.