Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping

• URL: http://arxiv.org/abs/2209.15165v2
• Date: Tue, 4 Oct 2022 16:10:28 GMT
• Title: Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping
• Authors: Aamir Mustafa, Param Hanji and Rafal K. Mantiuk
• Abstract summary: We show that information about the style can be distilled from collections of image pairs and encoded into a 2- or 3-dimensional vector. We show that such a network is more effective than PCA or VAE at normalizing image style in low-dimensional space.
• Score: 17.692674513446153
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Many image enhancement or editing operations, such as forward and inverse tone mapping or color grading, do not have a unique solution, but instead a range of solutions, each representing a different style. Despite this, existing learning-based methods attempt to learn a unique mapping, disregarding this style. In this work, we show that information about the style can be distilled from collections of image pairs and encoded into a 2- or 3-dimensional vector. This gives us not only an efficient representation but also an interpretable latent space for editing the image style. We represent the global color mapping between a pair of images as a custom normalizing flow, conditioned on a polynomial basis of the pixel color. We show that such a network is more effective than PCA or VAE at encoding image style in low-dimensional space and lets us obtain an accuracy close to 40 dB, which is about 7-10 dB improvement over the state-of-the-art methods.

Related papers

• PixelShuffler: A Simple Image Translation Through Pixel Rearrangement [0.0]
  Style transfer is a widely researched application of image-to-image translation, where the goal is to synthesize an image that combines the content of one image with the style of another. Existing state-of-the-art methods often rely on complex neural networks, including diffusion models and language models, to achieve high-quality style transfer. We propose a novel pixel shuffle method that addresses the image-to-image translation problem generally with a specific demonstrative application in style transfer.
  arXiv  Detail & Related papers  (2024-10-03T22:08:41Z)
• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• Learning Diverse Tone Styles for Image Retouching [73.60013618215328]
  We propose to learn diverse image retouching with normalizing flow-based architectures. A joint-training pipeline is composed of a style encoder, a conditional RetouchNet, and the image tone style normalizing flow (TSFlow) module. Our proposed method performs favorably against state-of-the-art methods and is effective in generating diverse results.
  arXiv  Detail & Related papers  (2022-07-12T09:49:21Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• CAMS: Color-Aware Multi-Style Transfer [46.550390398057985]
  Style transfer aims to manipulate the appearance of a source image, or "content" image, to share similar texture and colors of a target "style" image. A commonly used approach to assist in transferring styles is based on Gram matrix optimization. We propose a color-aware multi-style transfer method that generates aesthetically pleasing results while preserving the style-color correlation between style and generated images.
  arXiv  Detail & Related papers  (2021-06-26T01:15:09Z)
• TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations [35.9576572490994]
  We propose TransFill, a multi-homography transformed fusion method to fill the hole by referring to another source image that shares scene contents with the target image. We learn to adjust the color and apply a pixel-level warping to each homography-warped source image to make it more consistent with the target. Our method achieves state-of-the-art performance on pairs of images across a variety of wide baselines and color differences, and generalizes to user-provided image pairs.
  arXiv  Detail & Related papers  (2021-03-29T22:45:07Z)
• Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning [116.91819311885166]
  We propose a hierarchical semantic alignment strategy via expanding the views generated by a single image to textbfCross-samples and Multi-level representation. Our method, termed as CsMl, has the ability to integrate multi-level visual representations across samples in a robust way.
  arXiv  Detail & Related papers  (2020-12-04T17:26:24Z)
• Deep Line Art Video Colorization with a Few References [49.7139016311314]
  We propose a deep architecture to automatically color line art videos with the same color style as the given reference images. Our framework consists of a color transform network and a temporal constraint network. Our model can achieve even better coloring results by fine-tuning the parameters with only a small amount of samples.
  arXiv  Detail & Related papers  (2020-03-24T06:57:40Z)
• P$^2$-GAN: Efficient Style Transfer Using Single Style Image [2.703193151632043]
  Style transfer is a useful image synthesis technique that can re-render given image into another artistic style. Generative Adversarial Network (GAN) is a widely adopted framework toward this task for its better representation ability on local style patterns. In this paper, a novel Patch Permutation GAN (P$2$-GAN) network that can efficiently learn the stroke style from a single style image is proposed.
  arXiv  Detail & Related papers  (2020-01-21T12:08:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.