Instance-aware Image Colorization
- URL: http://arxiv.org/abs/2005.10825v1
- Date: Thu, 21 May 2020 17:59:23 GMT
- Title: Instance-aware Image Colorization
- Authors: Jheng-Wei Su, Hung-Kuo Chu, Jia-Bin Huang
- Abstract summary: In this paper, we propose a method for achieving instance-aware colorization.
Our network architecture leverages an off-the-shelf object detector to obtain cropped object images.
We use a similar network to extract the full-image features and apply a fusion module to predict the final colors.
- Score: 51.12040118366072
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image colorization is inherently an ill-posed problem with multi-modal
uncertainty. Previous methods leverage the deep neural network to map input
grayscale images to plausible color outputs directly. Although these
learning-based methods have shown impressive performance, they usually fail on
the input images that contain multiple objects. The leading cause is that
existing models perform learning and colorization on the entire image. In the
absence of a clear figure-ground separation, these models cannot effectively
locate and learn meaningful object-level semantics. In this paper, we propose a
method for achieving instance-aware colorization. Our network architecture
leverages an off-the-shelf object detector to obtain cropped object images and
uses an instance colorization network to extract object-level features. We use
a similar network to extract the full-image features and apply a fusion module
to full object-level and image-level features to predict the final colors. Both
colorization networks and fusion modules are learned from a large-scale
dataset. Experimental results show that our work outperforms existing methods
on different quality metrics and achieves state-of-the-art performance on image
colorization.
Related papers
- Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model.
We present an effective way to encode user strokes to enable precise local color manipulation.
We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
arXiv Detail & Related papers (2024-02-16T17:51:13Z) - Incorporating Ensemble and Transfer Learning For An End-To-End
Auto-Colorized Image Detection Model [0.0]
This paper presents a novel approach that combines the advantages of transfer and ensemble learning approaches to help reduce training time and resource requirements.
The proposed model shows promising results, with accuracy ranging from 94.55% to 99.13%.
arXiv Detail & Related papers (2023-09-25T19:22:57Z) - ParaColorizer: Realistic Image Colorization using Parallel Generative
Networks [1.7778609937758327]
Grayscale image colorization is a fascinating application of AI for information restoration.
We present a parallel GAN-based colorization framework.
We show the shortcomings of the non-perceptual evaluation metrics commonly used to assess multi-modal problems.
arXiv Detail & Related papers (2022-08-17T13:49:44Z) - Image Colorization using U-Net with Skip Connections and Fusion Layer on
Landscape Images [1.784933900656067]
We present a novel technique to automatically colorize grayscale images that combine the U-Net model and Fusion Layer features.
This approach allows the model to learn the colorization of images from pre-trained U-Net.
arXiv Detail & Related papers (2022-05-25T15:41:01Z) - TUCaN: Progressively Teaching Colourisation to Capsules [13.50327471049997]
We introduce a novel downsampling upsampling architecture named TUCaN (Tiny UCapsNet)
We pose the problem as a per pixel colour classification task that identifies colours as a bin in a quantized space.
To train the network, in contrast with the standard end to end learning method, we propose the progressive learning scheme to extract the context of objects.
arXiv Detail & Related papers (2021-06-29T08:44:15Z) - Collaboration among Image and Object Level Features for Image
Colourisation [25.60139324272782]
Image colourisation is an ill-posed problem, with multiple correct solutions which depend on the context and object instances present in the input datum.
Previous approaches attacked the problem either by requiring intense user interactions or by exploiting the ability of convolutional neural networks (CNNs) in learning image level (context) features.
We propose a single network, named UCapsNet, that separate image-level features obtained through convolutions and object-level features captured by means of capsules.
Then, by skip connections over different layers, we enforce collaboration between such disentangling factors to produce high quality and plausible image colourisation.
arXiv Detail & Related papers (2021-01-19T11:48:12Z) - Image Colorization: A Survey and Dataset [94.59768013860668]
This article presents a comprehensive survey of state-of-the-art deep learning-based image colorization techniques.
It categorizes the existing colorization techniques into seven classes and discusses important factors governing their performance.
We perform an extensive experimental evaluation of existing image colorization methods using both existing datasets and our proposed one.
arXiv Detail & Related papers (2020-08-25T01:22:52Z) - Semantic-driven Colorization [78.88814849391352]
Recent colorization works implicitly predict the semantic information while learning to colorize black-and-white images.
In this study, we simulate that human-like action to let our network first learn to understand the photo, then colorize it.
arXiv Detail & Related papers (2020-06-13T08:13:30Z) - Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
Contrastive learning has revolutionized unsupervised representation learning.
Current contrastive models are ineffective at localizing the foreground object.
We propose a data-driven approach for learning in variance to backgrounds.
arXiv Detail & Related papers (2020-04-14T16:29:42Z) - Learning to Structure an Image with Few Colors [59.34619548026885]
We propose a color quantization network, ColorCNN, which learns to structure the images from the classification loss in an end-to-end manner.
With only a 1-bit color space (i.e., two colors), the proposed network achieves 82.1% top-1 accuracy on the CIFAR10 dataset.
For applications, when encoded with PNG, the proposed color quantization shows superiority over other image compression methods in the extremely low bit-rate regime.
arXiv Detail & Related papers (2020-03-17T17:56:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.