Photo style transfer with consistency losses
- URL: http://arxiv.org/abs/2005.04408v1
- Date: Sat, 9 May 2020 09:58:06 GMT
- Title: Photo style transfer with consistency losses
- Authors: Xu Yao, Gilles Puy, Patrick P\'erez
- Abstract summary: We train a pair of deep convolution networks (convnets), each of which transfers the style of one photo to the other.
To enforce photorealism, we introduce a content preserving mechanism by combining a cycle-consistency loss with a self-consistency loss.
We show that retraining only a small subset of the network parameters can be sufficient to adapt these convnets to new styles.
- Score: 5.844015313757267
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We address the problem of style transfer between two photos and propose a new
way to preserve photorealism. Using the single pair of photos available as
input, we train a pair of deep convolution networks (convnets), each of which
transfers the style of one photo to the other. To enforce photorealism, we
introduce a content preserving mechanism by combining a cycle-consistency loss
with a self-consistency loss. Experimental results show that this method does
not suffer from typical artifacts observed in methods working in the same
settings. We then further analyze some properties of these trained convnets.
First, we notice that they can be used to stylize other unseen images with same
known style. Second, we show that retraining only a small subset of the network
parameters can be sufficient to adapt these convnets to new styles.
Related papers
- Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style
Transfer [38.957512116073616]
We propose a zero-shot contrastive loss for diffusion models that doesn't require additional fine-tuning or auxiliary networks.
Our method can generate images with the same semantic content as the source image in a zero-shot manner.
arXiv Detail & Related papers (2023-03-15T13:47:02Z) - ColoristaNet for Photorealistic Video Style Transfer [15.38024996795316]
Photorealistic style transfer aims to transfer the artistic style of an image onto an input image or video while keeping photorealism.
We propose a self-supervised style transfer framework, which contains a style removal part and a style restoration part.
Experiments demonstrate that ColoristaNet can achieve better stylization effects when compared with state-of-the-art algorithms.
arXiv Detail & Related papers (2022-12-19T04:49:26Z) - Improving the Latent Space of Image Style Transfer [24.37383949267162]
In some cases, the feature statistics from the pre-trained encoder may not be consistent with the visual style we perceived.
In such an inappropriate latent space, the objective function of the existing methods will be optimized in the wrong direction.
We propose two contrastive training schemes to get a refined encoder that is more suitable for this task.
arXiv Detail & Related papers (2022-05-24T15:13:01Z) - Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning.
Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
arXiv Detail & Related papers (2022-05-19T13:11:24Z) - Saliency Constrained Arbitrary Image Style Transfer using SIFT and DCNN [22.57205921266602]
When common neural style transfer methods are used, the textures and colors in the style image are usually transferred imperfectly to the content image.
This paper proposes a novel saliency constrained method to reduce or avoid such effects.
The experiments show that the saliency maps of source images can help find the correct matching and avoid artifacts.
arXiv Detail & Related papers (2022-01-14T09:00:55Z) - Contrastive Feature Loss for Image Prediction [55.373404869092866]
Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result.
We introduce an information theory based approach to measuring similarity between two images.
We show that our formulation boosts the perceptual realism of output images when used as a drop-in replacement for the L1 loss.
arXiv Detail & Related papers (2021-11-12T20:39:52Z) - The Spatially-Correlative Loss for Various Image Translation Tasks [69.62228639870114]
We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency.
Previous methods attempt this by using pixel-level cycle-consistency or feature-level matching losses.
We show distinct improvement over baseline models in all three modes of unpaired I2I translation: single-modal, multi-modal, and even single-image translation.
arXiv Detail & Related papers (2021-04-02T02:13:30Z) - You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing
Neural Network [63.2086502120071]
We study how to make deep learning achieve image dehazing without training on the ground-truth clean image (unsupervised) and a image collection (untrained)
An unsupervised neural network will avoid the intensive labor collection of hazy-clean image pairs, and an untrained model is a real'' single image dehazing approach.
Motivated by the layer disentanglement idea, we propose a novel method, called you only look yourself (textbfYOLY) which could be one of the first unsupervised and untrained neural networks for image dehazing.
arXiv Detail & Related papers (2020-06-30T14:05:47Z) - Unsupervised Landmark Learning from Unpaired Data [117.81440795184587]
Recent attempts for unsupervised landmark learning leverage synthesized image pairs that are similar in appearance but different in poses.
We propose a cross-image cycle consistency framework which applies the swapping-reconstruction strategy twice to obtain the final supervision.
Our proposed framework is shown to outperform strong baselines by a large margin.
arXiv Detail & Related papers (2020-06-29T13:57:20Z) - Self-Supervised Linear Motion Deblurring [112.75317069916579]
Deep convolutional neural networks are state-of-the-art for image deblurring.
We present a differentiable reblur model for self-supervised motion deblurring.
Our experiments demonstrate that self-supervised single image deblurring is really feasible.
arXiv Detail & Related papers (2020-02-10T20:15:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.