Towards Vivid and Diverse Image Colorization with Generative Color Prior

• URL: http://arxiv.org/abs/2108.08826v1
• Date: Thu, 19 Aug 2021 17:49:21 GMT
• Title: Towards Vivid and Diverse Image Colorization with Generative Color Prior
• Authors: Yanze Wu, Xintao Wang, Yu Li, Honglun Zhang, Xun Zhao, Ying Shan
• Abstract summary: Recent deep-learning-based methods could automatically colorize images at a low cost. We aim at recovering vivid colors by leveraging the rich and diverse color priors encapsulated in a pretrained Generative Adversarial Networks (GAN) Thanks to the powerful generative color prior and delicate designs, our method could produce vivid colors with a single forward pass.
• Score: 17.087464490162073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Colorization has attracted increasing interest in recent years. Classic reference-based methods usually rely on external color images for plausible results. A large image database or online search engine is inevitably required for retrieving such exemplars. Recent deep-learning-based methods could automatically colorize images at a low cost. However, unsatisfactory artifacts and incoherent colors are always accompanied. In this work, we aim at recovering vivid colors by leveraging the rich and diverse color priors encapsulated in a pretrained Generative Adversarial Networks (GAN). Specifically, we first "retrieve" matched features (similar to exemplars) via a GAN encoder and then incorporate these features into the colorization process with feature modulations. Thanks to the powerful generative color prior and delicate designs, our method could produce vivid colors with a single forward pass. Moreover, it is highly convenient to obtain diverse results by modifying GAN latent codes. Our method also inherits the merit of interpretable controls of GANs and could attain controllable and smooth transitions by walking through GAN latent space. Extensive experiments and user studies demonstrate that our method achieves superior performance than previous works.

Related papers

• Transforming Color: A Novel Image Colorization Method [8.041659727964305]
  This paper introduces a novel method for image colorization that utilizes a color transformer and generative adversarial networks (GANs) The proposed method integrates a transformer architecture to capture global information and a GAN framework to improve visual quality. Experimental results show that the proposed network significantly outperforms other state-of-the-art colorization techniques.
  arXiv  Detail & Related papers  (2024-10-07T07:23:42Z)
• Multimodal Semantic-Aware Automatic Colorization with Diffusion Prior [15.188673173327658]
  We leverage the extraordinary generative ability of the diffusion prior to synthesize color with plausible semantics. We adopt multimodal high-level semantic priors to help the model understand the image content and deliver saturated colors. A luminance-aware decoder is designed to restore details and enhance overall visual quality.
  arXiv  Detail & Related papers  (2024-04-25T15:28:22Z)
• Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
  Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model. We present an effective way to encode user strokes to enable precise local color manipulation. We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
  arXiv  Detail & Related papers  (2024-02-16T17:51:13Z)
• DiffColor: Toward High Fidelity Text-Guided Image Colorization with Diffusion Models [12.897939032560537]
  We propose a new method called DiffColor to recover vivid colors conditioned on a prompt text. We first fine-tune a pre-trained text-to-image model to generate colorized images using a CLIP-based contrastive loss. Then we try to obtain an optimized text embedding aligning the colorized image and the text prompt, and a fine-tuned diffusion model enabling high-quality image reconstruction. Our method can produce vivid and diverse colors with a few iterations, and keep the structure and background intact while having colors well-aligned with the target language guidance.
  arXiv  Detail & Related papers  (2023-08-03T09:38:35Z)
• DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders [19.560271615736212]
  DDColor is an end-to-end method with dual decoders for image colorization. Our approach includes a pixel decoder and a query-based color decoder. Our two decoders work together to establish correlations between color and multi-scale semantic representations.
  arXiv  Detail & Related papers  (2022-12-22T11:17:57Z)
• Name Your Colour For the Task: Artificially Discover Colour Naming via Colour Quantisation Transformer [62.75343115345667]
  We propose a novel colour quantisation transformer, CQFormer, that quantises colour space while maintaining machine recognition on the quantised images. We observe the consistent evolution pattern between our artificial colour system and basic colour terms across human languages. Our colour quantisation method also offers an efficient quantisation method that effectively compresses the image storage.
  arXiv  Detail & Related papers  (2022-12-07T03:39:18Z)
• PalGAN: Image Colorization with Palette Generative Adversarial Networks [51.59276436217957]
  We propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention. PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances.
  arXiv  Detail & Related papers  (2022-10-20T12:28:31Z)
• BigColor: Colorization using a Generative Color Prior for Natural Images [28.42665080958172]
  We propose BigColor, a novel colorization approach that provides vivid colorization for diverse in-the-wild images with complex structures. Our method enables robust colorization for diverse inputs in a single forward pass, supports arbitrary input resolutions, and provides multi-modal colorization results.
  arXiv  Detail & Related papers  (2022-07-20T06:36:46Z)
• Detecting Recolored Image by Spatial Correlation [60.08643417333974]
  Image recoloring is an emerging editing technique that can manipulate the color values of an image to give it a new style. In this paper, we explore a solution from the perspective of the spatial correlation, which exhibits the generic detection capability for both conventional and deep learning-based recoloring. Our method achieves the state-of-the-art detection accuracy on multiple benchmark datasets and exhibits well generalization for unknown types of recoloring methods.
  arXiv  Detail & Related papers  (2022-04-23T01:54:06Z)
• HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms [52.77252727786091]
  HistoGAN is a color histogram-based method for controlling GAN-generated images' colors. We show how to expand HistoGAN to recolor real images.
  arXiv  Detail & Related papers  (2020-11-23T21:14:19Z)
• Instance-aware Image Colorization [51.12040118366072]
  In this paper, we propose a method for achieving instance-aware colorization. Our network architecture leverages an off-the-shelf object detector to obtain cropped object images. We use a similar network to extract the full-image features and apply a fusion module to predict the final colors.
  arXiv  Detail & Related papers  (2020-05-21T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.