Generative Probabilistic Image Colorization

• URL: http://arxiv.org/abs/2109.14518v1
• Date: Wed, 29 Sep 2021 16:10:12 GMT
• Title: Generative Probabilistic Image Colorization
• Authors: Chie Furusawa, Shinya Kitaoka, Michael Li, Yuri Odagiri
• Abstract summary: We propose a diffusion-based generative process that trains a sequence of probabilistic models to reverse each step of noise corruption. Given a line-drawing image as input, our method suggests multiple candidate colorized images. Our proposed approach performed well not only on color-conditional image generation tasks, but also on some practical image completion and inpainting tasks.
• Score: 2.110198946293069
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose Generative Probabilistic Image Colorization, a diffusion-based generative process that trains a sequence of probabilistic models to reverse each step of noise corruption. Given a line-drawing image as input, our method suggests multiple candidate colorized images. Therefore, our method accounts for the ill-posed nature of the colorization problem. We conducted comprehensive experiments investigating the colorization of line-drawing images, report the influence of a score-based MCMC approach that corrects the marginal distribution of estimated samples, and further compare different combinations of models and the similarity of their generated images. Despite using only a relatively small training dataset, we experimentally develop a method to generate multiple diverse colorization candidates which avoids mode collapse and does not require any additional constraints, losses, or re-training with alternative training conditions. Our proposed approach performed well not only on color-conditional image generation tasks using biased initial values, but also on some practical image completion and inpainting tasks.

Related papers

• Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
  Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model. We present an effective way to encode user strokes to enable precise local color manipulation. We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
  arXiv  Detail & Related papers  (2024-02-16T17:51:13Z)
• ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text [5.675944597452309]
  We introduce two variations of an image-guided latent diffusion model utilizing different image tokens from the pre-trained CLIP image encoder. We propose corresponding manipulation methods to adjust their results sequentially using weighted text inputs.
  arXiv  Detail & Related papers  (2024-01-02T22:46:12Z)
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
  Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation. We present GradPaint, which steers the generation towards a globally coherent image. We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-09-18T09:36:24Z)
• PalGAN: Image Colorization with Palette Generative Adversarial Networks [51.59276436217957]
  We propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention. PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances.
  arXiv  Detail & Related papers  (2022-10-20T12:28:31Z)
• Exemplar-Based Image Colorization with A Learning Framework [7.793461393970992]
  We propose an automatic colorization method with a learning framework. It decouples the colorization process and learning process so as to generate various color styles for the same gray image. It achieves comparable performance against the state-of-the-art colorization algorithms.
  arXiv  Detail & Related papers  (2022-09-13T07:15:25Z)
• ParaColorizer: Realistic Image Colorization using Parallel Generative Networks [1.7778609937758327]
  Grayscale image colorization is a fascinating application of AI for information restoration. We present a parallel GAN-based colorization framework. We show the shortcomings of the non-perceptual evaluation metrics commonly used to assess multi-modal problems.
  arXiv  Detail & Related papers  (2022-08-17T13:49:44Z)
• Estimating Appearance Models for Image Segmentation via Tensor Factorization [0.0]
  We propose a new approach to directly estimate appearance models from the image without prior information on the underlying segmentation. Our method uses local high order color statistics from the image as an input to tensor factorization-based estimator for latent variable models. This approach is able to estimate models in multiregion images and automatically output the regions proportions without prior user interaction.
  arXiv  Detail & Related papers  (2022-08-16T17:21:00Z)
• Detecting Recolored Image by Spatial Correlation [60.08643417333974]
  Image recoloring is an emerging editing technique that can manipulate the color values of an image to give it a new style. In this paper, we explore a solution from the perspective of the spatial correlation, which exhibits the generic detection capability for both conventional and deep learning-based recoloring. Our method achieves the state-of-the-art detection accuracy on multiple benchmark datasets and exhibits well generalization for unknown types of recoloring methods.
  arXiv  Detail & Related papers  (2022-04-23T01:54:06Z)
• Mitigating Channel-wise Noise for Single Image Super Resolution [33.383282898248076]
  We propose to super-resolve noisy color images by considering the color channels jointly. Results demonstrate the super-resolving capability of the approach in real scenarios.
  arXiv  Detail & Related papers  (2021-12-14T17:45:15Z)
• Deblurring via Stochastic Refinement [85.42730934561101]
  We present an alternative framework for blind deblurring based on conditional diffusion models. Our method is competitive in terms of distortion metrics such as PSNR.
  arXiv  Detail & Related papers  (2021-12-05T04:36:09Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.