Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models

• URL: http://arxiv.org/abs/2403.08266v1
• Date: Wed, 13 Mar 2024 05:33:52 GMT
• Title: Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models
• Authors: Jian Lin, Xueting Liu, Chengze Li, Minshan Xie, Tien-Tsin Wong
• Abstract summary: We propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga. Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.
• Score: 26.010509997863196
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: While manga is a popular entertainment form, creating manga is tedious, especially adding screentones to the created sketch, namely manga screening. Unfortunately, there is no existing method that tailors for automatic manga screening, probably due to the difficulty of generating high-quality shaded high-frequency screentones. The classic manga screening approaches generally require user input to provide screentone exemplars or a reference manga image. The recent deep learning models enables the automatic generation by learning from a large-scale dataset. However, the state-of-the-art models still fail to generate high-quality shaded screentones due to the lack of a tailored model and high-quality manga training data. In this paper, we propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga based on the intensity guidance. Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.

Related papers

• The Manga Whisperer: Automatically Generating Transcriptions for Comics [55.544015596503726]
  We present a unified model, Magi, that is able to detect panels, text boxes and character boxes. We propose a novel approach that is able to sort the detected text boxes in their reading order and generate a dialogue transcript.
  arXiv  Detail & Related papers  (2024-01-18T18:59:09Z)
• inkn'hue: Enhancing Manga Colorization from Multiple Priors with Alignment Multi-Encoder VAE [0.0]
  We propose a specialized framework for manga colorization. We leverage established models for shading and vibrant coloring using a multi-encoder VAE. This structured workflow ensures clear and colorful results, with the option to incorporate reference images and manual hints.
  arXiv  Detail & Related papers  (2023-11-03T09:33:32Z)
• Deep Geometrized Cartoon Line Inbetweening [98.35956631655357]
  Inbetweening involves generating intermediate frames between two black-and-white line drawings. Existing frame methods that rely on matching and warping whole images are unsuitable for line inbetweening. We propose AnimeInbet, which geometrizes geometric line drawings into endpoints and reframes the inbetweening task as a graph fusion problem. Our method can effectively capture the sparsity and unique structure of line drawings while preserving the details during inbetweening.
  arXiv  Detail & Related papers  (2023-09-28T17:50:05Z)
• SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling [69.28254439393298]
  SketchMetaFace is a sketching system targeting amateur users to model high-fidelity 3D faces in minutes. We develop a novel learning-based method termed "Implicit and Depth Guided Mesh Modeling" (IDGMM) It fuses the advantages of mesh, implicit, and depth representations to achieve high-quality results with high efficiency.
  arXiv  Detail & Related papers  (2023-07-03T07:41:07Z)
• Screentone-Aware Manga Super-Resolution Using DeepLearning [3.0638744222997034]
  High-quality images can hinder transmission and affect the viewing experience. Traditional vectorization methods require a significant amount of manual parameter adjustment to process screentone. Super-resolution can convert low-resolution images to high-resolution images while maintaining low transmission rates and providing high-quality results.
  arXiv  Detail & Related papers  (2023-05-15T03:24:36Z)
• Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation [51.64832538714455]
  Existing studies only generate portraits in the 2D plane with fixed views, making the results less vivid. In this paper, we present Stereoscopic Simplified Sketch-to-Portrait (SSSP), which explores the possibility of creating Stereoscopic 3D-aware portraits. Our key insight is to design sketch-aware constraints that can fully exploit the prior knowledge of a tri-plane-based 3D-aware generative model.
  arXiv  Detail & Related papers  (2023-02-14T06:28:42Z)
• MARVEL: Raster Manga Vectorization via Primitive-wise Deep Reinforcement Learning [29.14983719525674]
  Manga is a fashionable Japanese-style comic form that is composed of black-and-white strokes and is generally displayed as images on digital devices. We propose MARVEL, a primitive-wise approach for vectorizing mangas by Deep Reinforcement Learning (DRL) Unlike previous learning-based methods which predict vector parameters for an entire image, MARVEL introduces a new perspective that regards an entire manga as a collection of basic primitivestextemdash stroke lines.
  arXiv  Detail & Related papers  (2021-10-10T15:52:38Z)
• Exploiting Aliasing for Manga Restoration [14.978972444431832]
  We propose an innovative two-stage method to restore quality bitonal manga from degraded ones. First, we predict the target resolution from the degraded manga via the Scale Estimation Network (SE-Net) Then, at the target resolution, we restore the region-wise bitonal screentones via the Manga Restoration Network (MR-Net)
  arXiv  Detail & Related papers  (2021-05-14T13:47:04Z)
• B\'ezierSketch: A generative model for scalable vector sketches [132.5223191478268]
  We present B'ezierSketch, a novel generative model for fully vector sketches that are automatically scalable and high-resolution. We first introduce a novel inverse graphics approach to stroke embedding that trains an encoder to embed each stroke to its best fit B'ezier curve. This enables us to treat sketches as short sequences of paramaterized strokes and thus train a recurrent sketch generator with greater capacity for longer sketches.
  arXiv  Detail & Related papers  (2020-07-04T21:30:52Z)
• MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing [27.99490750445691]
  We propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation. Inspired by how experienced manga artists draw manga, MangaGAN generates the geometric features of manga face by a designed GAN model. To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module.
  arXiv  Detail & Related papers  (2020-04-22T15:23:42Z)
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches [133.01690754567252]
  Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches. Deep Plastic Surgery is a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs.
  arXiv  Detail & Related papers  (2020-01-09T08:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.