ParaColorizer: Realistic Image Colorization using Parallel Generative
Networks
- URL: http://arxiv.org/abs/2208.08295v1
- Date: Wed, 17 Aug 2022 13:49:44 GMT
- Title: ParaColorizer: Realistic Image Colorization using Parallel Generative
Networks
- Authors: Himanshu Kumar, Abeer Banerjee, Sumeet Saurav, Sanjay Singh
- Abstract summary: Grayscale image colorization is a fascinating application of AI for information restoration.
We present a parallel GAN-based colorization framework.
We show the shortcomings of the non-perceptual evaluation metrics commonly used to assess multi-modal problems.
- Score: 1.7778609937758327
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Grayscale image colorization is a fascinating application of AI for
information restoration. The inherently ill-posed nature of the problem makes
it even more challenging since the outputs could be multi-modal. The
learning-based methods currently in use produce acceptable results for
straightforward cases but usually fail to restore the contextual information in
the absence of clear figure-ground separation. Also, the images suffer from
color bleeding and desaturated backgrounds since a single model trained on full
image features is insufficient for learning the diverse data modes. To address
these issues, we present a parallel GAN-based colorization framework. In our
approach, each separately tailored GAN pipeline colorizes the foreground (using
object-level features) or the background (using full-image features). The
foreground pipeline employs a Residual-UNet with self-attention as its
generator trained using the full-image features and the corresponding
object-level features from the COCO dataset. The background pipeline relies on
full-image features and additional training examples from the Places dataset.
We design a DenseFuse-based fusion network to obtain the final colorized image
by feature-based fusion of the parallelly generated outputs. We show the
shortcomings of the non-perceptual evaluation metrics commonly used to assess
multi-modal problems like image colorization and perform extensive performance
evaluation of our framework using multiple perceptual metrics. Our approach
outperforms most of the existing learning-based methods and produces results
comparable to the state-of-the-art. Further, we performed a runtime analysis
and obtained an average inference time of 24ms per image.
Related papers
- Detecting Recolored Image by Spatial Correlation [60.08643417333974]
Image recoloring is an emerging editing technique that can manipulate the color values of an image to give it a new style.
In this paper, we explore a solution from the perspective of the spatial correlation, which exhibits the generic detection capability for both conventional and deep learning-based recoloring.
Our method achieves the state-of-the-art detection accuracy on multiple benchmark datasets and exhibits well generalization for unknown types of recoloring methods.
arXiv Detail & Related papers (2022-04-23T01:54:06Z) - Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network.
We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
arXiv Detail & Related papers (2022-04-19T17:59:45Z) - Astronomical Image Colorization and upscaling with Generative
Adversarial Networks [0.0]
This research aims to provide an automated approach for the problem by focusing on a very specific domain of images, namely astronomical images.
We explore the usage of various models in two different color spaces, RGB and L*a*b.
The model produces visually appealing images which hallucinate high resolution, colorized data in these results which does not exist in the original image.
arXiv Detail & Related papers (2021-12-27T19:01:20Z) - Palette: Image-to-Image Diffusion Models [50.268441533631176]
We introduce Palette, a simple and general framework for image-to-image translation using conditional diffusion models.
On four challenging image-to-image translation tasks, Palette outperforms strong GAN and regression baselines.
We report several sample quality scores including FID, Inception Score, Classification Accuracy of a pre-trained ResNet-50, and Perceptual Distance against reference images.
arXiv Detail & Related papers (2021-11-10T17:49:29Z) - Generating Compositional Color Representations from Text [3.141061579698638]
Motivated by the fact that a significant fraction of user queries on an image search engine follow an (attribute, object) structure, we propose a generative adversarial network that generates color profiles for such bigrams.
We design our pipeline to learn composition - the ability to combine seen attributes and objects to unseen pairs.
arXiv Detail & Related papers (2021-09-22T01:37:13Z) - TUCaN: Progressively Teaching Colourisation to Capsules [13.50327471049997]
We introduce a novel downsampling upsampling architecture named TUCaN (Tiny UCapsNet)
We pose the problem as a per pixel colour classification task that identifies colours as a bin in a quantized space.
To train the network, in contrast with the standard end to end learning method, we propose the progressive learning scheme to extract the context of objects.
arXiv Detail & Related papers (2021-06-29T08:44:15Z) - Collaboration among Image and Object Level Features for Image
Colourisation [25.60139324272782]
Image colourisation is an ill-posed problem, with multiple correct solutions which depend on the context and object instances present in the input datum.
Previous approaches attacked the problem either by requiring intense user interactions or by exploiting the ability of convolutional neural networks (CNNs) in learning image level (context) features.
We propose a single network, named UCapsNet, that separate image-level features obtained through convolutions and object-level features captured by means of capsules.
Then, by skip connections over different layers, we enforce collaboration between such disentangling factors to produce high quality and plausible image colourisation.
arXiv Detail & Related papers (2021-01-19T11:48:12Z) - Image Colorization: A Survey and Dataset [94.59768013860668]
This article presents a comprehensive survey of state-of-the-art deep learning-based image colorization techniques.
It categorizes the existing colorization techniques into seven classes and discusses important factors governing their performance.
We perform an extensive experimental evaluation of existing image colorization methods using both existing datasets and our proposed one.
arXiv Detail & Related papers (2020-08-25T01:22:52Z) - Instance-aware Image Colorization [51.12040118366072]
In this paper, we propose a method for achieving instance-aware colorization.
Our network architecture leverages an off-the-shelf object detector to obtain cropped object images.
We use a similar network to extract the full-image features and apply a fusion module to predict the final colors.
arXiv Detail & Related papers (2020-05-21T17:59:23Z) - Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task.
We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network.
Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
arXiv Detail & Related papers (2020-03-15T11:04:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.