inkn'hue: Enhancing Manga Colorization from Multiple Priors with
Alignment Multi-Encoder VAE
- URL: http://arxiv.org/abs/2311.01804v2
- Date: Tue, 7 Nov 2023 15:06:50 GMT
- Title: inkn'hue: Enhancing Manga Colorization from Multiple Priors with
Alignment Multi-Encoder VAE
- Authors: Tawin Jiramahapokee
- Abstract summary: We propose a specialized framework for manga colorization.
We leverage established models for shading and vibrant coloring using a multi-encoder VAE.
This structured workflow ensures clear and colorful results, with the option to incorporate reference images and manual hints.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Manga, a form of Japanese comics and distinct visual storytelling, has
captivated readers worldwide. Traditionally presented in black and white,
manga's appeal lies in its ability to convey complex narratives and emotions
through intricate line art and shading. Yet, the desire to experience manga in
vibrant colors has sparked the pursuit of manga colorization, a task of
paramount significance for artists. However, existing methods, originally
designed for line art and sketches, face challenges when applied to manga.
These methods often fall short in achieving the desired results, leading to the
need for specialized manga-specific solutions. Existing approaches frequently
rely on a single training step or extensive manual artist intervention, which
can yield less satisfactory outcomes. To address these challenges, we propose a
specialized framework for manga colorization. Leveraging established models for
shading and vibrant coloring, our approach aligns both using a multi-encoder
VAE. This structured workflow ensures clear and colorful results, with the
option to incorporate reference images and manual hints.
Related papers
- Paint Bucket Colorization Using Anime Character Color Design Sheets [72.66788521378864]
We introduce inclusion matching, which allows the network to understand the relationships between segments.
Our network's training pipeline significantly improves performance in both colorization and consecutive frame colorization.
To support our network's training, we have developed a unique dataset named PaintBucket-Character.
arXiv Detail & Related papers (2024-10-25T09:33:27Z) - MangaUB: A Manga Understanding Benchmark for Large Multimodal Models [25.63892470012361]
Manga is a popular medium that combines stylized drawings and text to convey stories.
Recently, the adaptive nature of modern large multimodal models (LMMs) shows possibilities for more general approaches.
MangaUB is designed to assess the recognition and understanding of content shown in a single panel as well as conveyed across multiple panels.
arXiv Detail & Related papers (2024-07-26T18:21:30Z) - VIP: Versatile Image Outpainting Empowered by Multimodal Large Language Model [76.02314305164595]
This work presents a novel image outpainting framework that is capable of customizing the results according to the requirement of users.
We take advantage of a Multimodal Large Language Model (MLLM) that automatically extracts and organizes the corresponding textual descriptions of the masked and unmasked part of a given image.
In addition, a special Cross-Attention module, namely Center-Total-Surrounding (CTS), is elaborately designed to enhance further the the interaction between specific space regions of the image and corresponding parts of the text prompts.
arXiv Detail & Related papers (2024-06-03T07:14:19Z) - SketchDeco: Decorating B&W Sketches with Colour [80.90808879991182]
This paper introduces a novel approach to sketch colourisation, inspired by the universal childhood activity of colouring.
Striking a balance between precision and convenience, our method utilise region masks and colour palettes to allow intuitive user control.
arXiv Detail & Related papers (2024-05-29T02:53:59Z) - Learning Inclusion Matching for Animation Paint Bucket Colorization [76.4507878427755]
We introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments.
Our method features a two-stage pipeline that integrates a coarse color warping module with an inclusion matching module.
To facilitate the training of our network, we also develope a unique dataset, referred to as PaintBucket-Character.
arXiv Detail & Related papers (2024-03-27T08:32:48Z) - Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models [26.010509997863196]
We propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga.
Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.
arXiv Detail & Related papers (2024-03-13T05:33:52Z) - The Manga Whisperer: Automatically Generating Transcriptions for Comics [55.544015596503726]
We present a unified model, Magi, that is able to detect panels, text boxes and character boxes.
We propose a novel approach that is able to sort the detected text boxes in their reading order and generate a dialogue transcript.
arXiv Detail & Related papers (2024-01-18T18:59:09Z) - AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion
Models [24.94532405404846]
We propose a novel method called AnimeDiffusion using diffusion models that performs anime face line drawing colorization automatically.
We conduct an anime face line drawing colorization benchmark dataset, which contains 31696 training data and 579 testing data.
We demonstrate AnimeDiffusion outperforms state-of-the-art GANs-based models for anime face drawing colorization.
arXiv Detail & Related papers (2023-03-20T14:15:23Z) - Painting Style-Aware Manga Colorization Based on Generative Adversarial
Networks [9.495186818333815]
We propose a semi-automatic colorization method based on generative adversarial networks (GAN)
The proposed method takes a pair of a screen tone image and a flat colored image as input, and outputs a colorized image.
Experiments show that the proposed method achieves better performance than the existing alternatives.
arXiv Detail & Related papers (2021-07-16T15:00:28Z) - Multi-Density Sketch-to-Image Translation Network [65.4028451067947]
We propose the first multi-level density sketch-to-image translation framework, which allows the input sketch to cover a wide range from rough object outlines to micro structures.
Our method has been successfully verified on various datasets for different applications including face editing, multi-modal sketch-to-photo translation, and anime colorization.
arXiv Detail & Related papers (2020-06-18T16:21:04Z) - MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology
of Manga Drawing [27.99490750445691]
We propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation.
Inspired by how experienced manga artists draw manga, MangaGAN generates the geometric features of manga face by a designed GAN model.
To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module.
arXiv Detail & Related papers (2020-04-22T15:23:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.