Pluggable Style Representation Learning for Multi-Style Transfer

• URL: http://arxiv.org/abs/2503.20368v1
• Date: Wed, 26 Mar 2025 09:44:40 GMT
• Title: Pluggable Style Representation Learning for Multi-Style Transfer
• Authors: Hongda Liu, Longguang Wang, Weijun Guan, Ye Zhang, Yulan Guo,
• Abstract summary: We develop a style transfer framework by decoupling the style modeling and transferring.<n>For style modeling, we propose a style representation learning scheme to encode the style information into a compact representation.<n>For style transferring, we develop a style-aware multi-style transfer network (SaMST) to adapt to diverse styles using pluggable style representations.
• Score: 41.09041735653436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the high diversity of image styles, the scalability to various styles plays a critical role in real-world applications. To accommodate a large amount of styles, previous multi-style transfer approaches rely on enlarging the model size while arbitrary-style transfer methods utilize heavy backbones. However, the additional computational cost introduced by more model parameters hinders these methods to be deployed on resource-limited devices. To address this challenge, in this paper, we develop a style transfer framework by decoupling the style modeling and transferring. Specifically, for style modeling, we propose a style representation learning scheme to encode the style information into a compact representation. Then, for style transferring, we develop a style-aware multi-style transfer network (SaMST) to adapt to diverse styles using pluggable style representations. In this way, our framework is able to accommodate diverse image styles in the learned style representations without introducing additional overhead during inference, thereby maintaining efficiency. Experiments show that our style representation can extract accurate style information. Moreover, qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance in terms of both accuracy and efficiency. The codes are available in https://github.com/The-Learning-And-Vision-Atelier-LAVA/SaMST.

Related papers

• Ada-adapter:Fast Few-shot Style Personlization of Diffusion Model with Pre-trained Image Encoder [57.574544285878794]
  Ada-Adapter is a novel framework for few-shot style personalization of diffusion models. Our method enables efficient zero-shot style transfer utilizing a single reference image. We demonstrate the effectiveness of our approach on various artistic styles, including flat art, 3D rendering, and logo design.
  arXiv  Detail & Related papers  (2024-07-08T02:00:17Z)
• StyleShot: A Snapshot on Any Style [20.41380860802149]
  We show that, a good style representation is crucial and sufficient for generalized style transfer without test-time tuning.<n>We achieve this through constructing a style-aware encoder and a well-organized style dataset called StyleGallery.<n>We highlight that, our approach, named StyleShot, is simple yet effective in mimicking various desired styles, without test-time tuning.
  arXiv  Detail & Related papers  (2024-07-01T16:05:18Z)
• Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning [11.900404048019594]
  In this paper, we introduce an innovative technique to improve the quality of stylized images. Firstly, we propose Style Consistency Instance Normalization (SCIN), a method to refine the alignment between content and style features. In addition, we have developed an Instance-based Contrastive Learning (ICL) approach designed to understand relationships among various styles.
  arXiv  Detail & Related papers  (2024-04-21T08:52:22Z)
• StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model [64.26721402514957]
  We propose StylerDALLE, a style transfer method that uses natural language to describe abstract art styles. Specifically, we formulate the language-guided style transfer task as a non-autoregressive token sequence translation. To incorporate style information, we propose a Reinforcement Learning strategy with CLIP-based language supervision.
  arXiv  Detail & Related papers  (2023-03-16T12:44:44Z)
• A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning [84.8813842101747]
  Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework. We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
  arXiv  Detail & Related papers  (2023-03-09T04:35:00Z)
• StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning [89.86971464234533]
  Cross-Domain Few-Shot Learning (CD-FSL) is a recently emerging task that tackles few-shot learning across different domains. We propose a novel model-agnostic meta Style Adversarial training (StyleAdv) method together with a novel style adversarial attack method. Our method is gradually robust to the visual styles, thus boosting the generalization ability for novel target datasets.
  arXiv  Detail & Related papers  (2023-02-18T11:54:37Z)
• Style-Agnostic Reinforcement Learning [9.338454092492901]
  We present a novel method of learning style-agnostic representation using both style transfer and adversarial learning. Our method trains the actor with diverse image styles generated from an inherent adversarial style generator. We verify that our method achieves competitive or better performances than the state-of-the-art approaches on Procgen and Distracting Control Suite benchmarks.
  arXiv  Detail & Related papers  (2022-08-31T13:45:00Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Arbitrary Style Transfer via Multi-Adaptation Network [109.6765099732799]
  A desired style transfer, given a content image and referenced style painting, would render the content image with the color tone and vivid stroke patterns of the style painting. A new disentanglement loss function enables our network to extract main style patterns and exact content structures to adapt to various input images.
  arXiv  Detail & Related papers  (2020-05-27T08:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.