Exploring Palette based Color Guidance in Diffusion Models

• URL: http://arxiv.org/abs/2508.08754v1
• Date: Tue, 12 Aug 2025 09:02:10 GMT
• Title: Exploring Palette based Color Guidance in Diffusion Models
• Authors: Qianru Qiu, Jiafeng Mao, Xueting Wang,
• Abstract summary: We propose a novel approach to enhance color scheme control by integrating color palettes as a separate guidance mechanism alongside prompt instructions.<n>Our results demonstrate that incorporating palette guidance significantly improves the model's ability to generate images with desired color schemes.
• Score: 5.80330969550483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the advent of diffusion models, Text-to-Image (T2I) generation has seen substantial advancements. Current T2I models allow users to specify object colors using linguistic color names, and some methods aim to personalize color-object association through prompt learning. However, existing models struggle to provide comprehensive control over the color schemes of an entire image, especially for background elements and less prominent objects not explicitly mentioned in prompts. This paper proposes a novel approach to enhance color scheme control by integrating color palettes as a separate guidance mechanism alongside prompt instructions. We investigate the effectiveness of palette guidance by exploring various palette representation methods within a diffusion-based image colorization framework. To facilitate this exploration, we construct specialized palette-text-image datasets and conduct extensive quantitative and qualitative analyses. Our results demonstrate that incorporating palette guidance significantly improves the model's ability to generate images with desired color schemes, enabling a more controlled and refined colorization process.

Related papers

• ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation [4.714111142188893]
  We explore the use of pretrained Large Language Models (LLMs) and their commonsense reasoning capabilities for color recommendation.<n>Our approach primarily targeted color palette completion by recommending colors based on a set of given colors and accompanying context.<n>Our method can be extended to full palette generation, producing an entire color palette corresponding to a provided textual description.
  arXiv  Detail & Related papers  (2025-08-12T14:56:11Z)
• Instance-aware Image Colorization with Controllable Textual Descriptions and Segmentation Masks [60.495900243979754]
  Current mainstream image colorization models face issues such as color bleeding and color binding errors.<n>We propose a diffusion-based colorization method MT-Color to achieve precise instance-aware colorization with use-provided guidance.<n>We have created a specialized dataset for instance-level colorization tasks, GPT-color, by leveraging large visual language models on existing image datasets.
  arXiv  Detail & Related papers  (2025-05-13T16:13:06Z)
• Free-Lunch Color-Texture Disentanglement for Stylized Image Generation [58.406368812760256]
  This paper introduces the first tuning-free approach to achieve free-lunch color-texture disentanglement in stylized T2I generation.<n>We develop techniques for separating and extracting Color-Texture Embeddings (CTE) from individual color and texture reference images.<n>To ensure that the color palette of the generated image aligns closely with the color reference, we apply a whitening and coloring transformation.
  arXiv  Detail & Related papers  (2025-03-18T14:10:43Z)
• Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models [53.73253164099701]
  We introduce ColorWave, a training-free approach that achieves exact RGB-level color control in diffusion models without fine-tuning.<n>We demonstrate that ColorWave establishes a new paradigm for structured, color-consistent diffusion-based image synthesis.
  arXiv  Detail & Related papers  (2025-03-12T21:49:52Z)
• MangaNinja: Line Art Colorization with Precise Reference Following [84.2001766692797]
  MangaNinjia specializes in the task of reference-guided line art colorization.<n>We incorporate two thoughtful designs to ensure precise character detail transcription.<n>A patch shuffling module to facilitate correspondence learning between the reference color image and the target line art, and a point-driven control scheme to enable fine-grained color matching.
  arXiv  Detail & Related papers  (2025-01-14T18:59:55Z)
• ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglement [20.45850285936787]
  We propose to learn specific color prompts tailored to user-selected colors. Our method, denoted as ColorPeel, successfully assists the T2I models to peel off the novel color prompts. Our findings represent a significant step towards improving precision and versatility of T2I models.
  arXiv  Detail & Related papers  (2024-07-09T19:26:34Z)
• Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
  Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model. We present an effective way to encode user strokes to enable precise local color manipulation. We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
  arXiv  Detail & Related papers  (2024-02-16T17:51:13Z)
• Multimodal Color Recommendation in Vector Graphic Documents [14.287758028119788]
  We propose a multimodal masked color model that integrates both color and textual contexts to provide text-aware color recommendation for graphic documents. Our proposed model comprises self-attention networks to capture the relationships between colors in multiple palettes, and cross-attention networks that incorporate both color and CLIP-based text representations.
  arXiv  Detail & Related papers  (2023-08-08T08:17:39Z)
• Language-based Photo Color Adjustment for Graphic Designs [38.43984897069872]
  We introduce an interactive language-based approach for photo recoloring. Our model can predict the source colors and the target regions, and then recolor the target regions with the source colors based on the given language-based instruction.
  arXiv  Detail & Related papers  (2023-08-06T08:53:49Z)
• BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization [70.14893481468525]
  We present an effective BiSTNet to explore colors of reference exemplars and utilize them to help video colorization. We first establish the semantic correspondence between each frame and the reference exemplars in deep feature space to explore color information from reference exemplars. We develop a mixed expert block to extract semantic information for modeling the object boundaries of frames so that the semantic image prior can better guide the colorization process.
  arXiv  Detail & Related papers  (2022-12-05T13:47:15Z)
• PalGAN: Image Colorization with Palette Generative Adversarial Networks [51.59276436217957]
  We propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention. PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances.
  arXiv  Detail & Related papers  (2022-10-20T12:28:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.