Language-based Photo Color Adjustment for Graphic Designs
- URL: http://arxiv.org/abs/2308.03059v1
- Date: Sun, 6 Aug 2023 08:53:49 GMT
- Title: Language-based Photo Color Adjustment for Graphic Designs
- Authors: Zhenwei Wang, Nanxuan Zhao, Gerhard Hancke, Rynson W.H. Lau
- Abstract summary: We introduce an interactive language-based approach for photo recoloring.
Our model can predict the source colors and the target regions, and then recolor the target regions with the source colors based on the given language-based instruction.
- Score: 38.43984897069872
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Adjusting the photo color to associate with some design elements is an
essential way for a graphic design to effectively deliver its message and make
it aesthetically pleasing. However, existing tools and previous works face a
dilemma between the ease of use and level of expressiveness. To this end, we
introduce an interactive language-based approach for photo recoloring, which
provides an intuitive system that can assist both experts and novices on
graphic design. Given a graphic design containing a photo that needs to be
recolored, our model can predict the source colors and the target regions, and
then recolor the target regions with the source colors based on the given
language-based instruction. The multi-granularity of the instruction allows
diverse user intentions. The proposed novel task faces several unique
challenges, including: 1) color accuracy for recoloring with exactly the same
color from the target design element as specified by the user; 2)
multi-granularity instructions for parsing instructions correctly to generate a
specific result or multiple plausible ones; and 3) locality for recoloring in
semantically meaningful local regions to preserve original image semantics. To
address these challenges, we propose a model called LangRecol with two main
components: the language-based source color prediction module and the
semantic-palette-based photo recoloring module. We also introduce an approach
for generating a synthetic graphic design dataset with instructions to enable
model training. We evaluate our model via extensive experiments and user
studies. We also discuss several practical applications, showing the
effectiveness and practicality of our approach. Code and data for this paper
are at: https://zhenwwang.github.io/langrecol.
Related papers
- Paint Bucket Colorization Using Anime Character Color Design Sheets [72.66788521378864]
We introduce inclusion matching, which allows the network to understand the relationships between segments.
Our network's training pipeline significantly improves performance in both colorization and consecutive frame colorization.
To support our network's training, we have developed a unique dataset named PaintBucket-Character.
arXiv Detail & Related papers (2024-10-25T09:33:27Z) - ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglement [20.45850285936787]
We propose to learn specific color prompts tailored to user-selected colors.
Our method, denoted as ColorPeel, successfully assists the T2I models to peel off the novel color prompts.
Our findings represent a significant step towards improving precision and versatility of T2I models.
arXiv Detail & Related papers (2024-07-09T19:26:34Z) - VIP: Versatile Image Outpainting Empowered by Multimodal Large Language Model [76.02314305164595]
This work presents a novel image outpainting framework that is capable of customizing the results according to the requirement of users.
We take advantage of a Multimodal Large Language Model (MLLM) that automatically extracts and organizes the corresponding textual descriptions of the masked and unmasked part of a given image.
In addition, a special Cross-Attention module, namely Center-Total-Surrounding (CTS), is elaborately designed to enhance further the the interaction between specific space regions of the image and corresponding parts of the text prompts.
arXiv Detail & Related papers (2024-06-03T07:14:19Z) - Palette-based Color Transfer between Images [9.471264982229508]
We propose a new palette-based color transfer method that can automatically generate a new color scheme.
With a redesigned palette-based clustering method, pixels can be classified into different segments according to color distribution.
Our method exhibits significant advantages over peer methods in terms of natural realism, color consistency, generality, and robustness.
arXiv Detail & Related papers (2024-05-14T01:41:19Z) - Automatic Controllable Colorization via Imagination [55.489416987587305]
We propose a framework for automatic colorization that allows for iterative editing and modifications.
By understanding the content within a grayscale image, we utilize a pre-trained image generation model to generate multiple images that contain the same content.
These images serve as references for coloring, mimicking the process of human experts.
arXiv Detail & Related papers (2024-04-08T16:46:07Z) - Tuning-Free Image Customization with Image and Text Guidance [65.9504243633169]
We introduce a tuning-free framework for simultaneous text-image-guided image customization.
Our approach preserves the semantic features of the reference image subject while allowing modification of detailed attributes based on text descriptions.
Our approach outperforms previous methods in both human and quantitative evaluations.
arXiv Detail & Related papers (2024-03-19T11:48:35Z) - Fine-Tuning InstructPix2Pix for Advanced Image Colorization [3.4975669723257035]
This paper presents a novel approach to human image colorization by fine-tuning the InstructPix2Pix model.
We fine-tune the model using the IMDB-WIKI dataset, pairing black-and-white images with a diverse set of colorization prompts generated by ChatGPT.
After finetuning, our model outperforms the original InstructPix2Pix model on multiple metrics quantitatively.
arXiv Detail & Related papers (2023-12-08T01:36:49Z) - L-CAD: Language-based Colorization with Any-level Descriptions using
Diffusion Priors [62.80068955192816]
We propose a unified model to perform language-based colorization with any-level descriptions.
We leverage the pretrained cross-modality generative model for its robust language understanding and rich color priors.
With the proposed novel sampling strategy, our model achieves instance-aware colorization in diverse and complex scenarios.
arXiv Detail & Related papers (2023-05-24T14:57:42Z) - Color2Style: Real-Time Exemplar-Based Image Colorization with
Self-Reference Learning and Deep Feature Modulation [29.270149925368674]
We present a deep exemplar-based image colorization approach named Color2Style to resurrect grayscale image media by filling them with vibrant colors.
Our method exploits a simple yet effective deep feature modulation (DFM) module, which injects the color embeddings extracted from the reference image into the deep representations of the input grayscale image.
arXiv Detail & Related papers (2021-06-15T10:05:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.