Palette Aligned Image Diffusion

• URL: http://arxiv.org/abs/2509.02000v1
• Date: Tue, 02 Sep 2025 06:26:52 GMT
• Title: Palette Aligned Image Diffusion
• Authors: Elad Aharoni, Noy Porat, Dani Lischinski, Ariel Shamir,
• Abstract summary: Palette-Adapter is a novel method for conditioning text-to-image diffusion models on a user-specified color palette.<n>We introduce two scalar control parameters: histogram entropy and palette-to-histogram distance.<n>We show that it outperforms existing approaches in achieving both strong palette adherence and high image quality.
• Score: 31.91476258282606
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce the Palette-Adapter, a novel method for conditioning text-to-image diffusion models on a user-specified color palette. While palettes are a compact and intuitive tool widely used in creative workflows, they introduce significant ambiguity and instability when used for conditioning image generation. Our approach addresses this challenge by interpreting palettes as sparse histograms and introducing two scalar control parameters: histogram entropy and palette-to-histogram distance, which allow flexible control over the degree of palette adherence and color variation. We further introduce a negative histogram mechanism that allows users to suppress specific undesired hues, improving adherence to the intended palette under the standard classifier-free guidance mechanism. To ensure broad generalization across the color space, we train on a carefully curated dataset with balanced coverage of rare and common colors. Our method enables stable, semantically coherent generation across a wide range of palettes and prompts. We evaluate our method qualitatively, quantitatively, and through a user study, and show that it consistently outperforms existing approaches in achieving both strong palette adherence and high image quality.

Related papers

• Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer [58.94607850223466]
  We present Color3D, a highly adaptable framework for colorizing both static and dynamic 3D scenes from monochromatic inputs.<n>Our approach is able to preserve color diversity and steerability while ensuring cross-view and cross-time consistency.
  arXiv  Detail & Related papers  (2025-10-11T10:21:19Z)
• Exploring Palette based Color Guidance in Diffusion Models [5.80330969550483]
  We propose a novel approach to enhance color scheme control by integrating color palettes as a separate guidance mechanism alongside prompt instructions.<n>Our results demonstrate that incorporating palette guidance significantly improves the model's ability to generate images with desired color schemes.
  arXiv  Detail & Related papers  (2025-08-12T09:02:10Z)
• Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models [53.73253164099701]
  We introduce ColorWave, a training-free approach that achieves exact RGB-level color control in diffusion models without fine-tuning.<n>We demonstrate that ColorWave establishes a new paradigm for structured, color-consistent diffusion-based image synthesis.
  arXiv  Detail & Related papers  (2025-03-12T21:49:52Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• SketchDeco: Decorating B&W Sketches with Colour [80.90808879991182]
  This paper introduces a novel approach to sketch colourisation, inspired by the universal childhood activity of colouring. Striking a balance between precision and convenience, our method utilise region masks and colour palettes to allow intuitive user control.
  arXiv  Detail & Related papers  (2024-05-29T02:53:59Z)
• Palette-based Color Transfer between Images [9.471264982229508]
  We propose a new palette-based color transfer method that can automatically generate a new color scheme. With a redesigned palette-based clustering method, pixels can be classified into different segments according to color distribution. Our method exhibits significant advantages over peer methods in terms of natural realism, color consistency, generality, and robustness.
  arXiv  Detail & Related papers  (2024-05-14T01:41:19Z)
• Fine color guidance in diffusion models and its application to image compression at extremely low bitrates [9.17424462858218]
  This study addresses the challenge of, without training or fine-tuning, controlling the global color aspect of images generated with a diffusion model. We rewrite the guidance equations to ensure that the outputs are closer to a known color map, and this without hindering the quality of the generation.
  arXiv  Detail & Related papers  (2024-04-10T09:45:02Z)
• Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
  Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model. We present an effective way to encode user strokes to enable precise local color manipulation. We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
  arXiv  Detail & Related papers  (2024-02-16T17:51:13Z)
• Diffusing Colors: Image Colorization with Text Guided Diffusion [11.727899027933466]
  We present a novel image colorization framework that utilizes image diffusion techniques with granular text prompts. Our method provides a balance between automation and control, outperforming existing techniques in terms of visual quality and semantic coherence. Our approach holds potential particularly for color enhancement and historical image colorization.
  arXiv  Detail & Related papers  (2023-12-07T08:59:20Z)
• Dequantization and Color Transfer with Diffusion Models [5.228564799458042]
  quantized images offer easy abstraction for patch-based edits and palette transfer.<n>We show that our model can generate natural images that respect the color palette the user asked for.<n>Our method can be usefully extended to another practical edit: recoloring patches of an image while respecting the source texture.
  arXiv  Detail & Related papers  (2023-07-06T00:07:32Z)
• PalGAN: Image Colorization with Palette Generative Adversarial Networks [51.59276436217957]
  We propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention. PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances.
  arXiv  Detail & Related papers  (2022-10-20T12:28:31Z)
• Palette: Image-to-Image Diffusion Models [50.268441533631176]
  We introduce Palette, a simple and general framework for image-to-image translation using conditional diffusion models. On four challenging image-to-image translation tasks, Palette outperforms strong GAN and regression baselines. We report several sample quality scores including FID, Inception Score, Classification Accuracy of a pre-trained ResNet-50, and Perceptual Distance against reference images.
  arXiv  Detail & Related papers  (2021-11-10T17:49:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.