The Utility of Decorrelating Colour Spaces in Vector Quantised Variational Autoencoders

• URL: http://arxiv.org/abs/2009.14487v1
• Date: Wed, 30 Sep 2020 07:44:01 GMT
• Title: The Utility of Decorrelating Colour Spaces in Vector Quantised Variational Autoencoders
• Authors: Arash Akbarinia, Raquel Gil-Rodr\'iguez, Alban Flachot and Matteo Toscani
• Abstract summary: We propose colour space conversion to enforce a network learning structured representations. We trained several instances of VQ-VAE whose input is an image in one colour space, and its output in another.
• Score: 1.7792264784100689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vector quantised variational autoencoders (VQ-VAE) are characterised by three main components: 1) encoding visual data, 2) assigning $k$ different vectors in the so-called embedding space, and 3) decoding the learnt features. While images are often represented in RGB colour space, the specific organisation of colours in other spaces also offer interesting features, e.g. CIE L*a*b* decorrelates chromaticity into opponent axes. In this article, we propose colour space conversion, a simple quasi-unsupervised task, to enforce a network learning structured representations. To this end, we trained several instances of VQ-VAE whose input is an image in one colour space, and its output in another, e.g. from RGB to CIE L*a*b* (in total five colour spaces were considered). We examined the finite embedding space of trained networks in order to disentangle the colour representation in VQ-VAE models. Our analysis suggests that certain vectors encode hue and others luminance information. We further evaluated the quality of reconstructed images at low-level using pixel-wise colour metrics, and at high-level by inputting them to image classification and scene segmentation networks. We conducted experiments in three benchmark datasets: ImageNet, COCO and CelebA. Our results show, with respect to the baseline network (whose input and output are RGB), colour conversion to decorrelated spaces obtains 1-2 Delta-E lower colour difference and 5-10% higher classification accuracy. We also observed that the learnt embedding space is easier to interpret in colour opponent models.

Related papers

• MultiColor: Image Colorization by Learning from Multiple Color Spaces [4.738828630428634]
  MultiColor is a new learning-based approach to automatically colorize grayscale images. We employ a set of dedicated colorization modules for individual color space. With these predicted color channels representing various color spaces, a complementary network is designed to exploit the complementarity and generate pleasing and reasonable colorized images.
  arXiv  Detail & Related papers  (2024-08-08T02:34:41Z)
• Training Neural Networks on RAW and HDR Images for Restoration Tasks [59.41340420564656]
  In this work, we test approaches on three popular image restoration applications: denoising, deblurring, and single-image super-resolution. Our results indicate that neural networks train significantly better on HDR and RAW images represented in display color spaces. This small change to the training strategy can bring a very substantial gain in performance, up to 10-15 dB.
  arXiv  Detail & Related papers  (2023-12-06T17:47:16Z)
• UniGS: Unified Representation for Image Generation and Segmentation [105.08152635402858]
  We use a colormap to represent entity-level masks, addressing the challenge of varying entity numbers. Two novel modules, including the location-aware color palette and progressive dichotomy module, are proposed to support our mask representation.
  arXiv  Detail & Related papers  (2023-12-04T15:59:27Z)
• Color Equivariant Convolutional Networks [50.655443383582124]
  CNNs struggle if there is data imbalance between color variations introduced by accidental recording conditions. We propose Color Equivariant Convolutions ( CEConvs), a novel deep learning building block that enables shape feature sharing across the color spectrum. We demonstrate the benefits of CEConvs in terms of downstream performance to various tasks and improved robustness to color changes, including train-test distribution shifts.
  arXiv  Detail & Related papers  (2023-10-30T09:18:49Z)
• Name Your Colour For the Task: Artificially Discover Colour Naming via Colour Quantisation Transformer [62.75343115345667]
  We propose a novel colour quantisation transformer, CQFormer, that quantises colour space while maintaining machine recognition on the quantised images. We observe the consistent evolution pattern between our artificial colour system and basic colour terms across human languages. Our colour quantisation method also offers an efficient quantisation method that effectively compresses the image storage.
  arXiv  Detail & Related papers  (2022-12-07T03:39:18Z)
• Learning to Structure an Image with Few Colors and Beyond [59.34619548026885]
  We propose a color quantization network, ColorCNN, which learns to structure an image in limited color spaces by minimizing the classification loss. We introduce ColorCNN+, which supports multiple color space size configurations, and addresses the previous issues of poor recognition accuracy and undesirable visual fidelity under large color spaces. For potential applications, we show that ColorCNNs can be used as image compression methods for network recognition.
  arXiv  Detail & Related papers  (2022-08-17T17:59:15Z)
• Influence of Color Spaces for Deep Learning Image Colorization [2.3705923859070217]
  Existing colorization methods rely on different color spaces: RGB, YUV, Lab, etc. In this chapter, we aim to study their influence on the results obtained by training a deep neural network. We compare the results obtained with the same deep neural network architecture with RGB, YUV and Lab color spaces.
  arXiv  Detail & Related papers  (2022-04-06T14:14:07Z)
• Astronomical Image Colorization and upscaling with Generative Adversarial Networks [0.0]
  This research aims to provide an automated approach for the problem by focusing on a very specific domain of images, namely astronomical images. We explore the usage of various models in two different color spaces, RGB and L*a*b. The model produces visually appealing images which hallucinate high resolution, colorized data in these results which does not exist in the original image.
  arXiv  Detail & Related papers  (2021-12-27T19:01:20Z)
• Universal Deep Network for Steganalysis of Color Image based on Channel Representation [41.86330101334733]
  We design a universal color image steganalysis network (called UCNet) in spatial and JPEG domains. The proposed method includes preprocessing, convolutional, and classification modules. We conduct extensive experiments on ALASKA II to demonstrate that the proposed method can achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2021-11-24T02:22:13Z)
• Learning to Structure an Image with Few Colors [59.34619548026885]
  We propose a color quantization network, ColorCNN, which learns to structure the images from the classification loss in an end-to-end manner. With only a 1-bit color space (i.e., two colors), the proposed network achieves 82.1% top-1 accuracy on the CIFAR10 dataset. For applications, when encoded with PNG, the proposed color quantization shows superiority over other image compression methods in the extremely low bit-rate regime.
  arXiv  Detail & Related papers  (2020-03-17T17:56:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.