StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfer

• URL: http://arxiv.org/abs/2501.11319v1
• Date: Mon, 20 Jan 2025 07:45:42 GMT
• Title: StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfer
• Authors: Ruojun Xu, Weijie Xi, Xiaodi Wang, Yongbo Mao, Zach Cheng,
• Abstract summary: Training-free diffusion-based methods have achieved remarkable success in style transfer.<n>However, training-free methods often suffer from layout changes of original content and content leakage from style images.<n>We propose StyleSSP, which focuses on obtaining a better startpoint to address layout changes of original content and content leakage from style image.
• Score: 0.5740526382143176
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Training-free diffusion-based methods have achieved remarkable success in style transfer, eliminating the need for extensive training or fine-tuning. However, due to the lack of targeted training for style information extraction and constraints on the content image layout, training-free methods often suffer from layout changes of original content and content leakage from style images. Through a series of experiments, we discovered that an effective startpoint in the sampling stage significantly enhances the style transfer process. Based on this discovery, we propose StyleSSP, which focuses on obtaining a better startpoint to address layout changes of original content and content leakage from style image. StyleSSP comprises two key components: (1) Frequency Manipulation: To improve content preservation, we reduce the low-frequency components of the DDIM latent, allowing the sampling stage to pay more attention to the layout of content images; and (2) Negative Guidance via Inversion: To mitigate the content leakage from style image, we employ negative guidance in the inversion stage to ensure that the startpoint of the sampling stage is distanced from the content of style image. Experiments show that StyleSSP surpasses previous training-free style transfer baselines, particularly in preserving original content and minimizing the content leakage from style image.

Related papers

• ConsisLoRA: Enhancing Content and Style Consistency for LoRA-based Style Transfer [20.088714830700916]
  Style transfer involves transferring the style from a reference image to the content of a target image. Recent advancements in LoRA-based (Low-Rank Adaptation) methods have shown promise in effectively capturing the style of a single image. These approaches still face significant challenges such as content inconsistency, style misalignment, and content leakage.
  arXiv  Detail & Related papers  (2025-03-13T17:55:58Z)
• Inversion-Free Video Style Transfer with Trajectory Reset Attention Control and Content-Style Bridging [5.501345898413532]
  We introduce Trajectory Reset Attention Control (TRAC), a novel method that allows for high-quality style transfer. TRAC operates by resetting the denoising trajectory and enforcing attention control, thus enhancing content consistency. We present a tuning-free framework that offers a stable, flexible, and efficient solution for both image and video style transfer.
  arXiv  Detail & Related papers  (2025-03-10T14:18:43Z)
• DiffuseST: Unleashing the Capability of the Diffusion Model for Style Transfer [13.588643982359413]
  Style transfer aims to fuse the artistic representation of a style image with the structural information of a content image. Existing methods train specific networks or utilize pre-trained models to learn content and style features. We propose a novel and training-free approach for style transfer, combining textual embedding with spatial features.
  arXiv  Detail & Related papers  (2024-10-19T06:42:43Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generation [4.1177497612346]
  Style transfer is an inventive process designed to create an image that maintains the essence of the original while embracing the visual style of another. We introduce InstantStyle-Plus, an approach that prioritizes the integrity of the original content while seamlessly integrating the target style.
  arXiv  Detail & Related papers  (2024-06-30T18:05:33Z)
• InfoStyler: Disentanglement Information Bottleneck for Artistic Style Transfer [22.29381866838179]
  Artistic style transfer aims to transfer the style of an artwork to a photograph while maintaining its original overall content. We propose a novel information disentanglement method, named InfoStyler, to capture the minimal sufficient information for both content and style representations.
  arXiv  Detail & Related papers  (2023-07-30T13:38:56Z)
• StyleStegan: Leak-free Style Transfer Based on Feature Steganography [19.153040728118285]
  existing style transfer methods suffer from a serious content leakage issue. We propose a leak-free style transfer method based on feature steganography. The results demonstrate that StyleStegan successfully mitigates the content leakage issue in serial and reversible style transfer tasks.
  arXiv  Detail & Related papers  (2023-07-01T05:00:19Z)
• Edge Enhanced Image Style Transfer via Transformers [6.666912981102909]
  arbitrary image style transfer has attracted more and more attention. It is difficult to simultaneously keep well the trade-off between the content details and the style features. We present a new transformer-based method named STT for image style transfer and an edge loss.
  arXiv  Detail & Related papers  (2023-01-02T10:39:31Z)
• DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization [66.42741426640633]
  DiffStyler is a dual diffusion processing architecture to control the balance between the content and style of diffused results. We propose a content image-based learnable noise on which the reverse denoising process is based, enabling the stylization results to better preserve the structure information of the content image.
  arXiv  Detail & Related papers  (2022-11-19T12:30:44Z)
• StyTr^2: Unbiased Image Style Transfer with Transformers [59.34108877969477]
  The goal of image style transfer is to render an image with artistic features guided by a style reference while maintaining the original content. Traditional neural style transfer methods are usually biased and content leak can be observed by running several times of the style transfer process with the same reference image. We propose a transformer-based approach, namely StyTr2, to address this critical issue.
  arXiv  Detail & Related papers  (2021-05-30T15:57:09Z)
• Level-aware Haze Image Synthesis by Self-Supervised Content-Style Disentanglement [56.99803235546565]
  Key procedure of haze image translation through adversarial training lies in the disentanglement between the feature only involved in haze synthesis, i.e.style feature, and the feature representing the invariant semantic content, i.e. content feature.
  arXiv  Detail & Related papers  (2021-03-11T06:53:18Z)
• Arbitrary Style Transfer via Multi-Adaptation Network [109.6765099732799]
  A desired style transfer, given a content image and referenced style painting, would render the content image with the color tone and vivid stroke patterns of the style painting. A new disentanglement loss function enables our network to extract main style patterns and exact content structures to adapt to various input images.
  arXiv  Detail & Related papers  (2020-05-27T08:00:22Z)
• Parameter-Free Style Projection for Arbitrary Style Transfer [64.06126075460722]
  This paper proposes a new feature-level style transformation technique, named Style Projection, for parameter-free, fast, and effective content-style transformation. This paper further presents a real-time feed-forward model to leverage Style Projection for arbitrary image style transfer.
  arXiv  Detail & Related papers  (2020-03-17T13:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.