PalGAN: Image Colorization with Palette Generative Adversarial Networks
- URL: http://arxiv.org/abs/2210.11204v1
- Date: Thu, 20 Oct 2022 12:28:31 GMT
- Title: PalGAN: Image Colorization with Palette Generative Adversarial Networks
- Authors: Yi Wang, Menghan Xia, Lu Qi, Jing Shao, Yu Qiao
- Abstract summary: We propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention.
PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances.
- Score: 51.59276436217957
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Multimodal ambiguity and color bleeding remain challenging in colorization.
To tackle these problems, we propose a new GAN-based colorization approach
PalGAN, integrated with palette estimation and chromatic attention. To
circumvent the multimodality issue, we present a new colorization formulation
that estimates a probabilistic palette from the input gray image first, then
conducts color assignment conditioned on the palette through a generative
model. Further, we handle color bleeding with chromatic attention. It studies
color affinities by considering both semantic and intensity correlation. In
extensive experiments, PalGAN outperforms state-of-the-arts in quantitative
evaluation and visual comparison, delivering notable diverse, contrastive, and
edge-preserving appearances. With the palette design, our method enables color
transfer between images even with irrelevant contexts.
Related papers
- Palette-based Color Transfer between Images [9.471264982229508]
We propose a new palette-based color transfer method that can automatically generate a new color scheme.
With a redesigned palette-based clustering method, pixels can be classified into different segments according to color distribution.
Our method exhibits significant advantages over peer methods in terms of natural realism, color consistency, generality, and robustness.
arXiv Detail & Related papers (2024-05-14T01:41:19Z) - Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model.
We present an effective way to encode user strokes to enable precise local color manipulation.
We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
arXiv Detail & Related papers (2024-02-16T17:51:13Z) - ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text [5.675944597452309]
We introduce two variations of an image-guided latent diffusion model utilizing different image tokens from the pre-trained CLIP image encoder.
We propose corresponding manipulation methods to adjust their results sequentially using weighted text inputs.
arXiv Detail & Related papers (2024-01-02T22:46:12Z) - Diffusing Colors: Image Colorization with Text Guided Diffusion [11.727899027933466]
We present a novel image colorization framework that utilizes image diffusion techniques with granular text prompts.
Our method provides a balance between automation and control, outperforming existing techniques in terms of visual quality and semantic coherence.
Our approach holds potential particularly for color enhancement and historical image colorization.
arXiv Detail & Related papers (2023-12-07T08:59:20Z) - Multimodal Color Recommendation in Vector Graphic Documents [14.287758028119788]
We propose a multimodal masked color model that integrates both color and textual contexts to provide text-aware color recommendation for graphic documents.
Our proposed model comprises self-attention networks to capture the relationships between colors in multiple palettes, and cross-attention networks that incorporate both color and CLIP-based text representations.
arXiv Detail & Related papers (2023-08-08T08:17:39Z) - Dequantization and Color Transfer with Diffusion Models [5.228564799458042]
quantized images offer easy abstraction for patch-based edits and palette transfer.
We show that our model can generate natural images that respect the color palette the user asked for.
Our method can be usefully extended to another practical edit: recoloring patches of an image while respecting the source texture.
arXiv Detail & Related papers (2023-07-06T00:07:32Z) - Detecting Recolored Image by Spatial Correlation [60.08643417333974]
Image recoloring is an emerging editing technique that can manipulate the color values of an image to give it a new style.
In this paper, we explore a solution from the perspective of the spatial correlation, which exhibits the generic detection capability for both conventional and deep learning-based recoloring.
Our method achieves the state-of-the-art detection accuracy on multiple benchmark datasets and exhibits well generalization for unknown types of recoloring methods.
arXiv Detail & Related papers (2022-04-23T01:54:06Z) - Palette: Image-to-Image Diffusion Models [50.268441533631176]
We introduce Palette, a simple and general framework for image-to-image translation using conditional diffusion models.
On four challenging image-to-image translation tasks, Palette outperforms strong GAN and regression baselines.
We report several sample quality scores including FID, Inception Score, Classification Accuracy of a pre-trained ResNet-50, and Perceptual Distance against reference images.
arXiv Detail & Related papers (2021-11-10T17:49:29Z) - Underwater Image Enhancement via Medium Transmission-Guided Multi-Color
Space Embedding [88.46682991985907]
We present an underwater image enhancement network via medium transmission-guided multi-color space embedding, called Ucolor.
Our network can effectively improve the visual quality of underwater images by exploiting multiple color spaces embedding.
arXiv Detail & Related papers (2021-04-27T07:35:30Z) - Probabilistic Color Constancy [88.85103410035929]
We define a framework for estimating the illumination of a scene by weighting the contribution of different image regions.
The proposed method achieves competitive performance, compared to the state-of-the-art, on INTEL-TAU dataset.
arXiv Detail & Related papers (2020-05-06T11:03:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.