Retinex-guided Channel-grouping based Patch Swap for Arbitrary Style Transfer

• URL: http://arxiv.org/abs/2309.10528v1
• Date: Tue, 19 Sep 2023 11:13:56 GMT
• Title: Retinex-guided Channel-grouping based Patch Swap for Arbitrary Style Transfer
• Authors: Chang Liu, Yi Niu, Mingming Ma, Fu Li and Guangming Shi
• Abstract summary: The basic principle of the patch-matching based style transfer is to substitute the patches of the content image feature maps by the closest patches from the style image feature maps. Existing techniques treat the full-channel style feature patches as simple signal tensors and create new style feature patches via signal-level fusion. We propose a Retinex theory guided, channel-grouping based patch swap technique to solve the above challenges.
• Score: 54.25418866649519
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The basic principle of the patch-matching based style transfer is to substitute the patches of the content image feature maps by the closest patches from the style image feature maps. Since the finite features harvested from one single aesthetic style image are inadequate to represent the rich textures of the content natural image, existing techniques treat the full-channel style feature patches as simple signal tensors and create new style feature patches via signal-level fusion, which ignore the implicit diversities existed in style features and thus fail for generating better stylised results. In this paper, we propose a Retinex theory guided, channel-grouping based patch swap technique to solve the above challenges. Channel-grouping strategy groups the style feature maps into surface and texture channels, which prevents the winner-takes-all problem. Retinex theory based decomposition controls a more stable channel code rate generation. In addition, we provide complementary fusion and multi-scale generation strategy to prevent unexpected black area and over-stylised results respectively. Experimental results demonstrate that the proposed method outperforms the existing techniques in providing more style-consistent textures while keeping the content fidelity.

Related papers

• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• Region-controlled Style Transfer [3.588126599266807]
  We propose a training method that uses a loss function to constrain the style intensity in different regions. This method guides the transfer strength of style features in different regions based on the gradient relationship between style and content images. We also introduce a novel feature fusion method that linearly transforms content features to resemble style features while preserving their semantic relationships.
  arXiv  Detail & Related papers  (2023-10-24T09:11:34Z)
• Learning Graph Neural Networks for Image Style Transfer [131.73237185888215]
  State-of-the-art parametric and non-parametric style transfer approaches are prone to either distorted local style patterns due to global statistics alignment, or unpleasing artifacts resulting from patch mismatching. In this paper, we study a novel semi-parametric neural style transfer framework that alleviates the deficiency of both parametric and non-parametric stylization.
  arXiv  Detail & Related papers  (2022-07-24T07:41:31Z)
• SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization [71.85169368997738]
  Artistic style transfer aims to transfer the style characteristics of one image onto another image while retaining its content. Self-Attention-based approaches have tackled this issue with partial success but suffer from unwanted artifacts. This paper aims to combine the best of both worlds: self-attention and normalization.
  arXiv  Detail & Related papers  (2021-05-13T08:01:01Z)
• Region-adaptive Texture Enhancement for Detailed Person Image Synthesis [86.69934638569815]
  RATE-Net is a novel framework for synthesizing person images with sharp texture details. The proposed framework leverages an additional texture enhancing module to extract appearance information from the source image. Experiments conducted on DeepFashion benchmark dataset have demonstrated the superiority of our framework compared with existing networks.
  arXiv  Detail & Related papers  (2020-05-26T02:33:21Z)
• Manifold Alignment for Semantically Aligned Style Transfer [61.1274057338588]
  We make a new assumption that image features from the same semantic region form a manifold and an image with multiple semantic regions follows a multi-manifold distribution. Based on this assumption, the style transfer problem is formulated as aligning two multi-manifold distributions. The proposed framework allows semantically similar regions between the output and the style image share similar style patterns.
  arXiv  Detail & Related papers  (2020-05-21T16:52:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.