inkn'hue: Enhancing Manga Colorization from Multiple Priors with
Alignment Multi-Encoder VAE
- URL: http://arxiv.org/abs/2311.01804v2
- Date: Tue, 7 Nov 2023 15:06:50 GMT
- Title: inkn'hue: Enhancing Manga Colorization from Multiple Priors with
Alignment Multi-Encoder VAE
- Authors: Tawin Jiramahapokee
- Abstract summary: We propose a specialized framework for manga colorization.
We leverage established models for shading and vibrant coloring using a multi-encoder VAE.
This structured workflow ensures clear and colorful results, with the option to incorporate reference images and manual hints.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Manga, a form of Japanese comics and distinct visual storytelling, has
captivated readers worldwide. Traditionally presented in black and white,
manga's appeal lies in its ability to convey complex narratives and emotions
through intricate line art and shading. Yet, the desire to experience manga in
vibrant colors has sparked the pursuit of manga colorization, a task of
paramount significance for artists. However, existing methods, originally
designed for line art and sketches, face challenges when applied to manga.
These methods often fall short in achieving the desired results, leading to the
need for specialized manga-specific solutions. Existing approaches frequently
rely on a single training step or extensive manual artist intervention, which
can yield less satisfactory outcomes. To address these challenges, we propose a
specialized framework for manga colorization. Leveraging established models for
shading and vibrant coloring, our approach aligns both using a multi-encoder
VAE. This structured workflow ensures clear and colorful results, with the
option to incorporate reference images and manual hints.
Related papers
- VIP: Versatile Image Outpainting Empowered by Multimodal Large Language Model [76.02314305164595]
This work presents a novel image outpainting framework that is capable of customizing the results according to the requirement of users.
We take advantage of a Multimodal Large Language Model (MLLM) that automatically extracts and organizes the corresponding textual descriptions of the masked and unmasked part of a given image.
In addition, a special Cross-Attention module, namely Center-Total-Surrounding (CTS), is elaborately designed to enhance further the the interaction between specific space regions of the image and corresponding parts of the text prompts.
arXiv Detail & Related papers (2024-06-03T07:14:19Z) - SketchDeco: Decorating B&W Sketches with Colour [80.90808879991182]
This paper introduces a novel approach to sketch colourisation, inspired by the universal childhood activity of colouring.
Striking a balance between precision and convenience, our method utilise region masks and colour palettes to allow intuitive user control.
arXiv Detail & Related papers (2024-05-29T02:53:59Z) - Learning Inclusion Matching for Animation Paint Bucket Colorization [76.4507878427755]
We introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments.
Our method features a two-stage pipeline that integrates a coarse color warping module with an inclusion matching module.
To facilitate the training of our network, we also develope a unique dataset, referred to as PaintBucket-Character.
arXiv Detail & Related papers (2024-03-27T08:32:48Z) - Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models [26.010509997863196]
We propose a novel sketch-to-manga framework that first generates a color illustration from the sketch and then generates a screentoned manga.
Our method significantly outperforms existing methods in generating high-quality manga with shaded high-frequency screentones.
arXiv Detail & Related papers (2024-03-13T05:33:52Z) - The Manga Whisperer: Automatically Generating Transcriptions for Comics [55.544015596503726]
We present a unified model, Magi, that is able to detect panels, text boxes and character boxes.
We propose a novel approach that is able to sort the detected text boxes in their reading order and generate a dialogue transcript.
arXiv Detail & Related papers (2024-01-18T18:59:09Z) - AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion
Models [24.94532405404846]
We propose a novel method called AnimeDiffusion using diffusion models that performs anime face line drawing colorization automatically.
We conduct an anime face line drawing colorization benchmark dataset, which contains 31696 training data and 579 testing data.
We demonstrate AnimeDiffusion outperforms state-of-the-art GANs-based models for anime face drawing colorization.
arXiv Detail & Related papers (2023-03-20T14:15:23Z) - Unsupervised Manga Character Re-identification via Face-body and
Spatial-temporal Associated Clustering [21.696847342192072]
The artistic expression and stylistic limitations of manga pose many challenges to the re-identification problem.
Inspired by the idea that some content-related features may help clustering, we propose a Face-body and Spatial-temporal Associated Clustering method.
In the face-body combination module, a face-body graph is constructed to solve problems such as exaggeration and deformation in artistic creation.
In the spatial-temporal relationship correction module, we analyze the appearance features of characters and design a temporal-spatial-related triplet loss to fine-tune the clustering.
arXiv Detail & Related papers (2022-04-10T07:28:41Z) - Painting Style-Aware Manga Colorization Based on Generative Adversarial
Networks [9.495186818333815]
We propose a semi-automatic colorization method based on generative adversarial networks (GAN)
The proposed method takes a pair of a screen tone image and a flat colored image as input, and outputs a colorized image.
Experiments show that the proposed method achieves better performance than the existing alternatives.
arXiv Detail & Related papers (2021-07-16T15:00:28Z) - Multi-Density Sketch-to-Image Translation Network [65.4028451067947]
We propose the first multi-level density sketch-to-image translation framework, which allows the input sketch to cover a wide range from rough object outlines to micro structures.
Our method has been successfully verified on various datasets for different applications including face editing, multi-modal sketch-to-photo translation, and anime colorization.
arXiv Detail & Related papers (2020-06-18T16:21:04Z) - MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology
of Manga Drawing [27.99490750445691]
We propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation.
Inspired by how experienced manga artists draw manga, MangaGAN generates the geometric features of manga face by a designed GAN model.
To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module.
arXiv Detail & Related papers (2020-04-22T15:23:42Z) - Deep Plastic Surgery: Robust and Controllable Image Editing with
Human-Drawn Sketches [133.01690754567252]
Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches.
Deep Plastic Surgery is a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs.
arXiv Detail & Related papers (2020-01-09T08:57:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.