MARVEL: Raster Manga Vectorization via Primitive-wise Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2110.04830v2
• Date: Tue, 18 Jul 2023 21:13:25 GMT
• Title: MARVEL: Raster Manga Vectorization via Primitive-wise Deep Reinforcement Learning
• Authors: Hao Su, Jianwei Niu, Xuefeng Liu, Jiahe Cui, Ji Wan
• Abstract summary: Manga is a fashionable Japanese-style comic form that is composed of black-and-white strokes and is generally displayed as images on digital devices. We propose MARVEL, a primitive-wise approach for vectorizing mangas by Deep Reinforcement Learning (DRL) Unlike previous learning-based methods which predict vector parameters for an entire image, MARVEL introduces a new perspective that regards an entire manga as a collection of basic primitivestextemdash stroke lines.
• Score: 29.14983719525674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Manga is a fashionable Japanese-style comic form that is composed of black-and-white strokes and is generally displayed as raster images on digital devices. Typical mangas have simple textures, wide lines, and few color gradients, which are vectorizable natures to enjoy the merits of vector graphics, e.g., adaptive resolutions and small file sizes. In this paper, we propose MARVEL (MAnga's Raster to VEctor Learning), a primitive-wise approach for vectorizing raster mangas by Deep Reinforcement Learning (DRL). Unlike previous learning-based methods which predict vector parameters for an entire image, MARVEL introduces a new perspective that regards an entire manga as a collection of basic primitives\textemdash stroke lines, and designs a DRL model to decompose the target image into a primitive sequence for achieving accurate vectorization. To improve vectorization accuracies and decrease file sizes, we further propose a stroke accuracy reward to predict accurate stroke lines, and a pruning mechanism to avoid generating erroneous and repeated strokes. Extensive subjective and objective experiments show that our MARVEL can generate impressive results and reaches the state-of-the-art level. Our code is open-source at: https://github.com/SwordHolderSH/Mang2Vec.

Related papers

• Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision [20.325246638505714]
  We introduce GRIMOIRE, a text-guided generative model that learns to map images onto a discrete codebook by reconstructing them as vector shapes. Unlike existing models that require direct supervision from data, GRIMOIRE learns using only image supervision which opens up vector generative modeling to significantly more data.
  arXiv  Detail & Related papers  (2024-10-08T12:41:31Z)
• Segmentation-guided Layer-wise Image Vectorization with Gradient Fills [6.037332707968933]
  We propose a segmentation-guided vectorization framework to convert images into concise vector graphics with gradient fills. With the guidance of an embedded gradient-aware segmentation, our approach progressively appends gradient-filled B'ezier paths to the output.
  arXiv  Detail & Related papers  (2024-08-28T12:08:25Z)
• SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis [66.44553285020066]
  SuperSVG is a superpixel-based vectorization model that achieves fast and high-precision image vectorization. We propose a two-stage self-training framework, where a coarse-stage model is employed to reconstruct the main structure and a refinement-stage model is used for enriching the details. Experiments demonstrate the superior performance of our method in terms of reconstruction accuracy and inference time compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-06-14T07:43:23Z)
• Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models [26.010509997863196]
  We propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga. Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.
  arXiv  Detail & Related papers  (2024-03-13T05:33:52Z)
• Deep Geometrized Cartoon Line Inbetweening [98.35956631655357]
  Inbetweening involves generating intermediate frames between two black-and-white line drawings. Existing frame methods that rely on matching and warping whole images are unsuitable for line inbetweening. We propose AnimeInbet, which geometrizes geometric line drawings into endpoints and reframes the inbetweening task as a graph fusion problem. Our method can effectively capture the sparsity and unique structure of line drawings while preserving the details during inbetweening.
  arXiv  Detail & Related papers  (2023-09-28T17:50:05Z)
• Text-Guided Vector Graphics Customization [31.41266632288932]
  We propose a novel pipeline that generates high-quality customized vector graphics based on textual prompts. Our method harnesses the capabilities of large pre-trained text-to-image models. We evaluate our method using multiple metrics from vector-level, image-level and text-level perspectives.
  arXiv  Detail & Related papers  (2023-09-21T17:59:01Z)
• VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models [82.93345261434943]
  We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches.
  arXiv  Detail & Related papers  (2022-11-21T10:04:27Z)
• Towards Layer-wise Image Vectorization [57.26058135389497]
  We propose Layerwise Image Vectorization, namely LIVE, to convert images to SVGs and simultaneously maintain its image topology. Live generates compact forms with layer-wise structures that are semantically consistent with human perspective. Live initiates human editable SVGs for both designers and can be used in other applications.
  arXiv  Detail & Related papers  (2022-06-09T17:55:02Z)
• Cloud2Curve: Generation and Vectorization of Parametric Sketches [109.02932608241227]
  We present Cloud2Curve, a generative model for scalable high-resolution vector sketches. We evaluate the generation and vectorization capabilities of our model on Quick, Draw! and KMNIST datasets.
  arXiv  Detail & Related papers  (2021-03-29T12:09:42Z)
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation [217.86315551526235]
  We propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and manipulation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool.
  arXiv  Detail & Related papers  (2020-07-22T09:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.