SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network

• URL: http://arxiv.org/abs/2011.11377v1
• Date: Mon, 23 Nov 2020 13:06:54 GMT
• Title: SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network
• Authors: Yuzhi Zhao, Lai-Man Po, Kwok-Wai Cheung, Wing-Yin Yu, Yasar Abbas Ur Rehman
• Abstract summary: We propose a fully automatic Saliency Map-guided Colorization with Generative Adversarial Network (SCGAN) framework. It jointly predicts the colorization and saliency map to minimize semantic confusion and color bleeding. Experimental results show that SCGAN can generate more reasonable colorized images than state-of-the-art techniques.
• Score: 16.906813829260553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given a grayscale photograph, the colorization system estimates a visually plausible colorful image. Conventional methods often use semantics to colorize grayscale images. However, in these methods, only classification semantic information is embedded, resulting in semantic confusion and color bleeding in the final colorized image. To address these issues, we propose a fully automatic Saliency Map-guided Colorization with Generative Adversarial Network (SCGAN) framework. It jointly predicts the colorization and saliency map to minimize semantic confusion and color bleeding in the colorized image. Since the global features from pre-trained VGG-16-Gray network are embedded to the colorization encoder, the proposed SCGAN can be trained with much less data than state-of-the-art methods to achieve perceptually reasonable colorization. In addition, we propose a novel saliency map-based guidance method. Branches of the colorization decoder are used to predict the saliency map as a proxy target. Moreover, two hierarchical discriminators are utilized for the generated colorization and saliency map, respectively, in order to strengthen visual perception performance. The proposed system is evaluated on ImageNet validation set. Experimental results show that SCGAN can generate more reasonable colorized images than state-of-the-art techniques.

Related papers

• Transforming Color: A Novel Image Colorization Method [8.041659727964305]
  This paper introduces a novel method for image colorization that utilizes a color transformer and generative adversarial networks (GANs) The proposed method integrates a transformer architecture to capture global information and a GAN framework to improve visual quality. Experimental results show that the proposed network significantly outperforms other state-of-the-art colorization techniques.
  arXiv  Detail & Related papers  (2024-10-07T07:23:42Z)
• Automatic Controllable Colorization via Imagination [55.489416987587305]
  We propose a framework for automatic colorization that allows for iterative editing and modifications. By understanding the content within a grayscale image, we utilize a pre-trained image generation model to generate multiple images that contain the same content. These images serve as references for coloring, mimicking the process of human experts.
  arXiv  Detail & Related papers  (2024-04-08T16:46:07Z)
• SPDGAN: A Generative Adversarial Network based on SPD Manifold Learning for Automatic Image Colorization [1.220743263007369]
  We propose a fully automatic colorization approach based on Symmetric Positive Definite (SPD) Manifold Learning with a generative adversarial network (SPDGAN) Our model establishes an adversarial game between two discriminators and a generator. Its goal is to generate fake colorized images without losing color information across layers through residual connections.
  arXiv  Detail & Related papers  (2023-12-21T00:52:01Z)
• Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization [23.301799487207035]
  Exemplar-based colorization approaches rely on reference image to provide plausible colors for target gray-scale image. We propose Semantic-Sparse Colorization Network (SSCN) to transfer both the global image style and semantic-related colors to the gray-scale image. Our network can perfectly balance the global and local colors while alleviating the ambiguous matching problem.
  arXiv  Detail & Related papers  (2021-12-02T15:35:10Z)
• HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms [52.77252727786091]
  HistoGAN is a color histogram-based method for controlling GAN-generated images' colors. We show how to expand HistoGAN to recolor real images.
  arXiv  Detail & Related papers  (2020-11-23T21:14:19Z)
• Image Colorization: A Survey and Dataset [94.59768013860668]
  This article presents a comprehensive survey of state-of-the-art deep learning-based image colorization techniques. It categorizes the existing colorization techniques into seven classes and discusses important factors governing their performance. We perform an extensive experimental evaluation of existing image colorization methods using both existing datasets and our proposed one.
  arXiv  Detail & Related papers  (2020-08-25T01:22:52Z)
• Controllable Image Synthesis via SegVAE [89.04391680233493]
  A semantic map is commonly used intermediate representation for conditional image generation. In this work, we specifically target at generating semantic maps given a label-set consisting of desired categories. The proposed framework, SegVAE, synthesizes semantic maps in an iterative manner using conditional variational autoencoder.
  arXiv  Detail & Related papers  (2020-07-16T15:18:53Z)
• Semantic-driven Colorization [78.88814849391352]
  Recent colorization works implicitly predict the semantic information while learning to colorize black-and-white images. In this study, we simulate that human-like action to let our network first learn to understand the photo, then colorize it.
  arXiv  Detail & Related papers  (2020-06-13T08:13:30Z)
• Learning to Structure an Image with Few Colors [59.34619548026885]
  We propose a color quantization network, ColorCNN, which learns to structure the images from the classification loss in an end-to-end manner. With only a 1-bit color space (i.e., two colors), the proposed network achieves 82.1% top-1 accuracy on the CIFAR10 dataset. For applications, when encoded with PNG, the proposed color quantization shows superiority over other image compression methods in the extremely low bit-rate regime.
  arXiv  Detail & Related papers  (2020-03-17T17:56:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.