Saliency Constrained Arbitrary Image Style Transfer using SIFT and DCNN
- URL: http://arxiv.org/abs/2201.05346v1
- Date: Fri, 14 Jan 2022 09:00:55 GMT
- Title: Saliency Constrained Arbitrary Image Style Transfer using SIFT and DCNN
- Authors: HuiHuang Zhao, Yaonan Wang and Yuhua Li
- Abstract summary: When common neural style transfer methods are used, the textures and colors in the style image are usually transferred imperfectly to the content image.
This paper proposes a novel saliency constrained method to reduce or avoid such effects.
The experiments show that the saliency maps of source images can help find the correct matching and avoid artifacts.
- Score: 22.57205921266602
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper develops a new image synthesis approach to transfer an example
image (style image) to other images (content images) by using Deep
Convolutional Neural Networks (DCNN) model. When common neural style transfer
methods are used, the textures and colors in the style image are usually
transferred imperfectly to the content image, or some visible errors are
generated. This paper proposes a novel saliency constrained method to reduce or
avoid such effects. It first evaluates some existing saliency detection methods
to select the most suitable one for use in our method. The selected saliency
detection method is used to detect the object in the style image, corresponding
to the object of the content image with the same saliency. In addition, aim to
solve the problem that the size or resolution is different in the style image
and content, the scale-invariant feature transform is used to generate a series
of style images and content images which can be used to generate more feature
maps for patches matching. It then proposes a new loss function combining the
saliency loss, style loss and content loss, adding gradient of saliency
constraint into style transfer in iterations. Finally the source images and
saliency detection results are utilized as multichannel input to an improved
deep CNN framework for style transfer. The experiments show that the saliency
maps of source images can help find the correct matching and avoid artifacts.
Experimental results on different kind of images demonstrate that our method
outperforms nine representative methods from recent publications and has good
robustness.
Related papers
- PixelShuffler: A Simple Image Translation Through Pixel Rearrangement [0.0]
Style transfer is a widely researched application of image-to-image translation, where the goal is to synthesize an image that combines the content of one image with the style of another.
Existing state-of-the-art methods often rely on complex neural networks, including diffusion models and language models, to achieve high-quality style transfer.
We propose a novel pixel shuffle method that addresses the image-to-image translation problem generally with a specific demonstrative application in style transfer.
arXiv Detail & Related papers (2024-10-03T22:08:41Z) - Neural Style Transfer for Vector Graphics [3.8983556368110226]
Style transfer between vector images has not been considered.
Applying standard content and style losses insignificantly changes the vector image drawing style.
New method based on differentiableization can change the color and shape parameters of the content image corresponding to the drawing of the style image.
arXiv Detail & Related papers (2023-03-06T16:57:45Z) - Diffusion-based Image Translation using Disentangled Style and Content
Representation [51.188396199083336]
Diffusion-based image translation guided by semantic texts or a single target image has enabled flexible style transfer.
It is often difficult to maintain the original content of the image during the reverse diffusion.
We present a novel diffusion-based unsupervised image translation method using disentangled style and content representation.
Our experimental results show that the proposed method outperforms state-of-the-art baseline models in both text-guided and image-guided translation tasks.
arXiv Detail & Related papers (2022-09-30T06:44:37Z) - Learning Diverse Tone Styles for Image Retouching [73.60013618215328]
We propose to learn diverse image retouching with normalizing flow-based architectures.
A joint-training pipeline is composed of a style encoder, a conditional RetouchNet, and the image tone style normalizing flow (TSFlow) module.
Our proposed method performs favorably against state-of-the-art methods and is effective in generating diverse results.
arXiv Detail & Related papers (2022-07-12T09:49:21Z) - Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning.
Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
arXiv Detail & Related papers (2022-05-19T13:11:24Z) - Controllable Person Image Synthesis with Spatially-Adaptive Warped
Normalization [72.65828901909708]
Controllable person image generation aims to produce realistic human images with desirable attributes.
We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters.
We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
arXiv Detail & Related papers (2021-05-31T07:07:44Z) - A Hierarchical Transformation-Discriminating Generative Model for Few
Shot Anomaly Detection [93.38607559281601]
We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image.
The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
arXiv Detail & Related papers (2021-04-29T17:49:48Z) - Image Restoration by Deep Projected GSURE [115.57142046076164]
Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution.
We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
arXiv Detail & Related papers (2021-02-04T08:52:46Z) - P$^2$-GAN: Efficient Style Transfer Using Single Style Image [2.703193151632043]
Style transfer is a useful image synthesis technique that can re-render given image into another artistic style.
Generative Adversarial Network (GAN) is a widely adopted framework toward this task for its better representation ability on local style patterns.
In this paper, a novel Patch Permutation GAN (P$2$-GAN) network that can efficiently learn the stroke style from a single style image is proposed.
arXiv Detail & Related papers (2020-01-21T12:08:08Z) - Learning Transformation-Aware Embeddings for Image Forensics [15.484408315588569]
Image Provenance Analysis aims at discovering relationships among different manipulated image versions that share content.
One of the main sub-problems for provenance analysis that has not yet been addressed directly is the edit ordering of images that share full content or are near-duplicates.
This paper introduces a novel deep learning-based approach to provide a plausible ordering to images that have been generated from a single image through transformations.
arXiv Detail & Related papers (2020-01-13T22:01:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.