GIST: Towards Photorealistic Style Transfer via Multiscale Geometric Representations

• URL: http://arxiv.org/abs/2412.02214v1
• Date: Tue, 03 Dec 2024 07:05:39 GMT
• Title: GIST: Towards Photorealistic Style Transfer via Multiscale Geometric Representations
• Authors: Renan A. Rojas-Gomez, Minh N. Do,
• Abstract summary: GIST replaces the standard Neural Style Transfer autoencoding framework with a multiscale image expansion. Our method matches multiresolution and multidirectional representations such as Wavelets and Contourlets by solving an optimal transport problem.
• Score: 9.514509577589449
• License:
• Abstract: State-of-the-art Style Transfer methods often leverage pre-trained encoders optimized for discriminative tasks, which may not be ideal for image synthesis. This can result in significant artifacts and loss of photorealism. Motivated by the ability of multiscale geometric image representations to capture fine-grained details and global structure, we propose GIST: Geometric-based Image Style Transfer, a novel Style Transfer technique that exploits the geometric properties of content and style images. GIST replaces the standard Neural Style Transfer autoencoding framework with a multiscale image expansion, preserving scene details without the need for post-processing or training. Our method matches multiresolution and multidirectional representations such as Wavelets and Contourlets by solving an optimal transport problem, leading to an efficient texture transferring. Experiments show that GIST is on-par or outperforms recent photorealistic Style Transfer approaches while significantly reducing the processing time with no model training.

Related papers

• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Universal Photorealistic Style Transfer: A Lightweight and Adaptive Approach [18.46446576915766]
  Photorealistic style transfer aims to apply stylization while preserving the realism and structure of input content. We propose a Universal Photorealistic Style Transfer framework that delivers accurate photorealistic style transfer on high-resolution images and videos.
  arXiv  Detail & Related papers  (2023-09-18T07:04:40Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• ColoristaNet for Photorealistic Video Style Transfer [15.38024996795316]
  Photorealistic style transfer aims to transfer the artistic style of an image onto an input image or video while keeping photorealism. We propose a self-supervised style transfer framework, which contains a style removal part and a style restoration part. Experiments demonstrate that ColoristaNet can achieve better stylization effects when compared with state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2022-12-19T04:49:26Z)
• Learning Diverse Tone Styles for Image Retouching [73.60013618215328]
  We propose to learn diverse image retouching with normalizing flow-based architectures. A joint-training pipeline is composed of a style encoder, a conditional RetouchNet, and the image tone style normalizing flow (TSFlow) module. Our proposed method performs favorably against state-of-the-art methods and is effective in generating diverse results.
  arXiv  Detail & Related papers  (2022-07-12T09:49:21Z)
• Deep Translation Prior: Test-time Training for Photorealistic Style Transfer [36.82737412912885]
  Recent techniques to solve photorealistic style transfer within deep convolutional neural networks (CNNs) generally require intensive training from large-scale datasets. We propose a novel framework, dubbed Deep Translation Prior (DTP), to accomplish photorealistic style transfer through test-time training on given input image pair with untrained networks.
  arXiv  Detail & Related papers  (2021-12-12T04:54:27Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• TSIT: A Simple and Versatile Framework for Image-to-Image Translation [103.92203013154403]
  We introduce a simple and versatile framework for image-to-image translation. We provide a carefully designed two-stream generative model with newly proposed feature transformations. This allows multi-scale semantic structure information and style representation to be effectively captured and fused by the network. A systematic study compares the proposed method with several state-of-the-art task-specific baselines, verifying its effectiveness in both perceptual quality and quantitative evaluations.
  arXiv  Detail & Related papers  (2020-07-23T15:34:06Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)
• Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer [18.455002563426262]
  Photorealistic style transfer is the task of transferring the artistic style of an image onto a content target, producing a result that is plausibly taken with a camera. Recent approaches, based on deep neural networks, produce impressive results but are either too slow to run at practical resolutions, or still contain objectionable artifacts. We propose a new end-to-end model for photorealistic style transfer that is both fast and inherently generates photorealistic results.
  arXiv  Detail & Related papers  (2020-04-23T03:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.