Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfer

• URL: http://arxiv.org/abs/2304.11818v1
• Date: Mon, 24 Apr 2023 04:46:39 GMT
• Title: Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfer
• Authors: Hao Tang, Songhua Liu, Tianwei Lin, Shaoli Huang, Fu Li, Dongliang He, Xinchao Wang
• Abstract summary: In this paper, we devise a novel Transformer model termed as emphMaster specifically for style transfer. In the proposed model, different Transformer layers share a common group of parameters, which (1) reduces the total number of parameters, (2) leads to more robust training convergence, and (3) is readily to control the degree of stylization. Experiments demonstrate the superiority of Master under both zero-shot and few-shot style transfer settings.
• Score: 83.1333306079676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformer-based models achieve favorable performance in artistic style transfer recently thanks to its global receptive field and powerful multi-head/layer attention operations. Nevertheless, the over-paramerized multi-layer structure increases parameters significantly and thus presents a heavy burden for training. Moreover, for the task of style transfer, vanilla Transformer that fuses content and style features by residual connections is prone to content-wise distortion. In this paper, we devise a novel Transformer model termed as \emph{Master} specifically for style transfer. On the one hand, in the proposed model, different Transformer layers share a common group of parameters, which (1) reduces the total number of parameters, (2) leads to more robust training convergence, and (3) is readily to control the degree of stylization via tuning the number of stacked layers freely during inference. On the other hand, different from the vanilla version, we adopt a learnable scaling operation on content features before content-style feature interaction, which better preserves the original similarity between a pair of content features while ensuring the stylization quality. We also propose a novel meta learning scheme for the proposed model so that it can not only work in the typical setting of arbitrary style transfer, but also adaptable to the few-shot setting, by only fine-tuning the Transformer encoder layer in the few-shot stage for one specific style. Text-guided few-shot style transfer is firstly achieved with the proposed framework. Extensive experiments demonstrate the superiority of Master under both zero-shot and few-shot style transfer settings.

Related papers

• Puff-Net: Efficient Style Transfer with Pure Content and Style Feature Fusion Network [32.12413686394824]
  Style transfer aims to render an image with the artistic features of a style image, while maintaining the original structure. It is difficult for CNN-based methods to handle global information and long-range dependencies between input images. We propose a novel network termed Puff-Net, i.e., pure content and style feature fusion network.
  arXiv  Detail & Related papers  (2024-05-30T07:41:07Z)
• Diffusion-based Human Motion Style Transfer with Semantic Guidance [23.600154466988073]
  We propose a novel framework for few-shot style transfer learning based on the diffusion model. In the first stage, we pre-train a diffusion-based text-to-motion model as a generative prior. In the second stage, based on the single style example, we fine-tune the pre-trained diffusion model in a few-shot manner to make it capable of style transfer.
  arXiv  Detail & Related papers  (2024-03-20T05:52:11Z)
• STEER: Unified Style Transfer with Expert Reinforcement [71.3995732115262]
  STEER: Unified Style Transfer with Expert Reinforcement, is a unified frame-work developed to overcome the challenge of limited parallel data for style transfer. We show STEER is robust, maintaining its style transfer capabilities on out-of-domain data, and surpassing nearly all baselines across various styles.
  arXiv  Detail & Related papers  (2023-11-13T09:02:30Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• Line Search-Based Feature Transformation for Fast, Stable, and Tunable Content-Style Control in Photorealistic Style Transfer [26.657485176782934]
  Photorealistic style transfer is the task of synthesizing a realistic-looking image when adapting the content from one image to appear in the style of another image. Modern models embed a transformation that fuses features describing the content image and style image and then decodes the resulting feature into a stylized image. We introduce a general-purpose transformation that enables controlling the balance between how much content is preserved and the strength of the infused style.
  arXiv  Detail & Related papers  (2022-10-12T08:05:49Z)
• CCPL: Contrastive Coherence Preserving Loss for Versatile Style Transfer [58.020470877242865]
  We devise a universally versatile style transfer method capable of performing artistic, photo-realistic, and video style transfer jointly. We make a mild and reasonable assumption that global inconsistency is dominated by local inconsistencies and devise a generic Contrastive Coherence Preserving Loss (CCPL) applied to local patches. CCPL can preserve the coherence of the content source during style transfer without degrading stylization.
  arXiv  Detail & Related papers  (2022-07-11T12:09:41Z)
• StyTr^2: Unbiased Image Style Transfer with Transformers [59.34108877969477]
  The goal of image style transfer is to render an image with artistic features guided by a style reference while maintaining the original content. Traditional neural style transfer methods are usually biased and content leak can be observed by running several times of the style transfer process with the same reference image. We propose a transformer-based approach, namely StyTr2, to address this critical issue.
  arXiv  Detail & Related papers  (2021-05-30T15:57:09Z)
• Parameter-Free Style Projection for Arbitrary Style Transfer [64.06126075460722]
  This paper proposes a new feature-level style transformation technique, named Style Projection, for parameter-free, fast, and effective content-style transformation. This paper further presents a real-time feed-forward model to leverage Style Projection for arbitrary image style transfer.
  arXiv  Detail & Related papers  (2020-03-17T13:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.