Saliency Constrained Arbitrary Image Style Transfer using SIFT and DCNN

• URL: http://arxiv.org/abs/2201.05346v1
• Date: Fri, 14 Jan 2022 09:00:55 GMT
• Title: Saliency Constrained Arbitrary Image Style Transfer using SIFT and DCNN
• Authors: HuiHuang Zhao, Yaonan Wang and Yuhua Li
• Abstract summary: When common neural style transfer methods are used, the textures and colors in the style image are usually transferred imperfectly to the content image. This paper proposes a novel saliency constrained method to reduce or avoid such effects. The experiments show that the saliency maps of source images can help find the correct matching and avoid artifacts.
• Score: 22.57205921266602
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper develops a new image synthesis approach to transfer an example image (style image) to other images (content images) by using Deep Convolutional Neural Networks (DCNN) model. When common neural style transfer methods are used, the textures and colors in the style image are usually transferred imperfectly to the content image, or some visible errors are generated. This paper proposes a novel saliency constrained method to reduce or avoid such effects. It first evaluates some existing saliency detection methods to select the most suitable one for use in our method. The selected saliency detection method is used to detect the object in the style image, corresponding to the object of the content image with the same saliency. In addition, aim to solve the problem that the size or resolution is different in the style image and content, the scale-invariant feature transform is used to generate a series of style images and content images which can be used to generate more feature maps for patches matching. It then proposes a new loss function combining the saliency loss, style loss and content loss, adding gradient of saliency constraint into style transfer in iterations. Finally the source images and saliency detection results are utilized as multichannel input to an improved deep CNN framework for style transfer. The experiments show that the saliency maps of source images can help find the correct matching and avoid artifacts. Experimental results on different kind of images demonstrate that our method outperforms nine representative methods from recent publications and has good robustness.

Related papers

• Neural Style Transfer for Vector Graphics [3.8983556368110226]
  Style transfer between vector images has not been considered. Applying standard content and style losses insignificantly changes the vector image drawing style. New method based on differentiableization can change the color and shape parameters of the content image corresponding to the drawing of the style image.
  arXiv  Detail & Related papers  (2023-03-06T16:57:45Z)
• Diffusion-based Image Translation using Disentangled Style and Content Representation [51.188396199083336]
  Diffusion-based image translation guided by semantic texts or a single target image has enabled flexible style transfer. It is often difficult to maintain the original content of the image during the reverse diffusion. We present a novel diffusion-based unsupervised image translation method using disentangled style and content representation. Our experimental results show that the proposed method outperforms state-of-the-art baseline models in both text-guided and image-guided translation tasks.
  arXiv  Detail & Related papers  (2022-09-30T06:44:37Z)
• Learning Diverse Tone Styles for Image Retouching [73.60013618215328]
  We propose to learn diverse image retouching with normalizing flow-based architectures. A joint-training pipeline is composed of a style encoder, a conditional RetouchNet, and the image tone style normalizing flow (TSFlow) module. Our proposed method performs favorably against state-of-the-art methods and is effective in generating diverse results.
  arXiv  Detail & Related papers  (2022-07-12T09:49:21Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• Image Restoration by Deep Projected GSURE [115.57142046076164]
  Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution. We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
  arXiv  Detail & Related papers  (2021-02-04T08:52:46Z)
• P$^2$-GAN: Efficient Style Transfer Using Single Style Image [2.703193151632043]
  Style transfer is a useful image synthesis technique that can re-render given image into another artistic style. Generative Adversarial Network (GAN) is a widely adopted framework toward this task for its better representation ability on local style patterns. In this paper, a novel Patch Permutation GAN (P$2$-GAN) network that can efficiently learn the stroke style from a single style image is proposed.
  arXiv  Detail & Related papers  (2020-01-21T12:08:08Z)
• Learning Transformation-Aware Embeddings for Image Forensics [15.484408315588569]
  Image Provenance Analysis aims at discovering relationships among different manipulated image versions that share content. One of the main sub-problems for provenance analysis that has not yet been addressed directly is the edit ordering of images that share full content or are near-duplicates. This paper introduces a novel deep learning-based approach to provide a plausible ordering to images that have been generated from a single image through transformations.
  arXiv  Detail & Related papers  (2020-01-13T22:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.