SCGAN: Saliency Map-guided Colorization with Generative Adversarial
Network
- URL: http://arxiv.org/abs/2011.11377v1
- Date: Mon, 23 Nov 2020 13:06:54 GMT
- Title: SCGAN: Saliency Map-guided Colorization with Generative Adversarial
Network
- Authors: Yuzhi Zhao, Lai-Man Po, Kwok-Wai Cheung, Wing-Yin Yu, Yasar Abbas Ur
Rehman
- Abstract summary: We propose a fully automatic Saliency Map-guided Colorization with Generative Adversarial Network (SCGAN) framework.
It jointly predicts the colorization and saliency map to minimize semantic confusion and color bleeding.
Experimental results show that SCGAN can generate more reasonable colorized images than state-of-the-art techniques.
- Score: 16.906813829260553
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Given a grayscale photograph, the colorization system estimates a visually
plausible colorful image. Conventional methods often use semantics to colorize
grayscale images. However, in these methods, only classification semantic
information is embedded, resulting in semantic confusion and color bleeding in
the final colorized image. To address these issues, we propose a fully
automatic Saliency Map-guided Colorization with Generative Adversarial Network
(SCGAN) framework. It jointly predicts the colorization and saliency map to
minimize semantic confusion and color bleeding in the colorized image. Since
the global features from pre-trained VGG-16-Gray network are embedded to the
colorization encoder, the proposed SCGAN can be trained with much less data
than state-of-the-art methods to achieve perceptually reasonable colorization.
In addition, we propose a novel saliency map-based guidance method. Branches of
the colorization decoder are used to predict the saliency map as a proxy
target. Moreover, two hierarchical discriminators are utilized for the
generated colorization and saliency map, respectively, in order to strengthen
visual perception performance. The proposed system is evaluated on ImageNet
validation set. Experimental results show that SCGAN can generate more
reasonable colorized images than state-of-the-art techniques.
Related papers
- Automatic Controllable Colorization via Imagination [55.489416987587305]
We propose a framework for automatic colorization that allows for iterative editing and modifications.
By understanding the content within a grayscale image, we utilize a pre-trained image generation model to generate multiple images that contain the same content.
These images serve as references for coloring, mimicking the process of human experts.
arXiv Detail & Related papers (2024-04-08T16:46:07Z) - SPDGAN: A Generative Adversarial Network based on SPD Manifold Learning
for Automatic Image Colorization [1.220743263007369]
We propose a fully automatic colorization approach based on Symmetric Positive Definite (SPD) Manifold Learning with a generative adversarial network (SPDGAN)
Our model establishes an adversarial game between two discriminators and a generator. Its goal is to generate fake colorized images without losing color information across layers through residual connections.
arXiv Detail & Related papers (2023-12-21T00:52:01Z) - Semantic-Sparse Colorization Network for Deep Exemplar-based
Colorization [23.301799487207035]
Exemplar-based colorization approaches rely on reference image to provide plausible colors for target gray-scale image.
We propose Semantic-Sparse Colorization Network (SSCN) to transfer both the global image style and semantic-related colors to the gray-scale image.
Our network can perfectly balance the global and local colors while alleviating the ambiguous matching problem.
arXiv Detail & Related papers (2021-12-02T15:35:10Z) - HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color
Histograms [52.77252727786091]
HistoGAN is a color histogram-based method for controlling GAN-generated images' colors.
We show how to expand HistoGAN to recolor real images.
arXiv Detail & Related papers (2020-11-23T21:14:19Z) - Visualizing Color-wise Saliency of Black-Box Image Classification Models [0.0]
A classification result given by an advanced method, including deep learning, is often hard to interpret.
We propose MC-RISE (Multi-Color RISE), which is an enhancement of RISE to take color information into account in an explanation.
Our method not only shows the saliency of each pixel in a given image as the original RISE does, but the significance of color components of each pixel; a saliency map with color information is useful especially in the domain where the color information matters.
arXiv Detail & Related papers (2020-10-06T04:27:18Z) - Image Colorization: A Survey and Dataset [78.89573261114428]
This article presents a comprehensive survey of state-of-the-art deep learning-based image colorization techniques.
It categorizes the existing colorization techniques into seven classes and discusses important factors governing their performance.
Using the existing datasets and our new one, we perform an extensive experimental evaluation of existing image colorization methods.
arXiv Detail & Related papers (2020-08-25T01:22:52Z) - Controllable Image Synthesis via SegVAE [89.04391680233493]
A semantic map is commonly used intermediate representation for conditional image generation.
In this work, we specifically target at generating semantic maps given a label-set consisting of desired categories.
The proposed framework, SegVAE, synthesizes semantic maps in an iterative manner using conditional variational autoencoder.
arXiv Detail & Related papers (2020-07-16T15:18:53Z) - Semantic-driven Colorization [78.88814849391352]
Recent colorization works implicitly predict the semantic information while learning to colorize black-and-white images.
In this study, we simulate that human-like action to let our network first learn to understand the photo, then colorize it.
arXiv Detail & Related papers (2020-06-13T08:13:30Z) - Learning to Structure an Image with Few Colors [59.34619548026885]
We propose a color quantization network, ColorCNN, which learns to structure the images from the classification loss in an end-to-end manner.
With only a 1-bit color space (i.e., two colors), the proposed network achieves 82.1% top-1 accuracy on the CIFAR10 dataset.
For applications, when encoded with PNG, the proposed color quantization shows superiority over other image compression methods in the extremely low bit-rate regime.
arXiv Detail & Related papers (2020-03-17T17:56:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.