LEAST: "Local" text-conditioned image style transfer

• URL: http://arxiv.org/abs/2405.16330v1
• Date: Sat, 25 May 2024 19:06:17 GMT
• Title: LEAST: "Local" text-conditioned image style transfer
• Authors: Silky Singh, Surgan Jandial, Simra Shahid, Abhinav Java,
• Abstract summary: Text-conditioned style transfer enables users to communicate their desired artistic styles through text descriptions. We evaluate the text-conditioned image editing and style transfer techniques on their fine-grained understanding of user prompts for precise "local" style transfer.
• Score: 2.47996065798589
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Text-conditioned style transfer enables users to communicate their desired artistic styles through text descriptions, offering a new and expressive means of achieving stylization. In this work, we evaluate the text-conditioned image editing and style transfer techniques on their fine-grained understanding of user prompts for precise "local" style transfer. We find that current methods fail to accomplish localized style transfers effectively, either failing to localize style transfer to certain regions in the image, or distorting the content and structure of the input image. To this end, we carefully design an end-to-end pipeline that guarantees local style transfer according to users' intent. Further, we substantiate the effectiveness of our approach through quantitative and qualitative analysis. The project code is available at: https://github.com/silky1708/local-style-transfer.

Related papers

• Bridging Text and Image for Artist Style Transfer via Contrastive Learning [21.962361974579036]
  We propose a Contrastive Learning for Artistic Style Transfer (CLAST) to control arbitrary style transfer. We introduce a supervised contrastive training strategy to effectively extract style descriptions from the image-text model. We also propose a novel and efficient adaLN based state space models that explore style-content fusion.
  arXiv  Detail & Related papers  (2024-10-12T15:27:57Z)
• StyleMamba : State Space Model for Efficient Text-driven Image Style Transfer [9.010012117838725]
  StyleMamba is an efficient image style transfer framework that translates text prompts into corresponding visual styles. Existing text-guided stylization requires hundreds of training iterations and takes a lot of computing resources.
  arXiv  Detail & Related papers  (2024-05-08T12:57:53Z)
• ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer [57.6482608202409]
  Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. We introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
  arXiv  Detail & Related papers  (2023-08-29T17:36:02Z)
• Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate [58.83278629019384]
  Style transfer aims to render the style of a given image for style reference to another given image for content reference. Existing approaches either apply the holistic style of the style image in a global manner, or migrate local colors and textures of the style image to the content counterparts in a pre-defined way. We propose Any-to-Any Style Transfer, which enables users to interactively select styles of regions in the style image and apply them to the prescribed content regions.
  arXiv  Detail & Related papers  (2023-04-19T15:15:36Z)
• StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model [64.26721402514957]
  We propose StylerDALLE, a style transfer method that uses natural language to describe abstract art styles. Specifically, we formulate the language-guided style transfer task as a non-autoregressive token sequence translation. To incorporate style information, we propose a Reinforcement Learning strategy with CLIP-based language supervision.
  arXiv  Detail & Related papers  (2023-03-16T12:44:44Z)
• ITstyler: Image-optimized Text-based Style Transfer [25.60521982742093]
  We present a text-based style transfer method that does not require optimization at the inference stage. Specifically, we convert text input to the style space of the pre-trained VGG network to realize a more effective style swap. Our method can transfer arbitrary new styles of text input in real-time and synthesize high-quality artistic images.
  arXiv  Detail & Related papers  (2023-01-26T03:08:43Z)
• DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization [66.42741426640633]
  DiffStyler is a dual diffusion processing architecture to control the balance between the content and style of diffused results. We propose a content image-based learnable noise on which the reverse denoising process is based, enabling the stylization results to better preserve the structure information of the content image.
  arXiv  Detail & Related papers  (2022-11-19T12:30:44Z)
• StoryTrans: Non-Parallel Story Author-Style Transfer with Discourse Representations and Content Enhancing [73.81778485157234]
  Long texts usually involve more complicated author linguistic preferences such as discourse structures than sentences. We formulate the task of non-parallel story author-style transfer, which requires transferring an input story into a specified author style. We use an additional training objective to disentangle stylistic features from the learned discourse representation to prevent the model from degenerating to an auto-encoder.
  arXiv  Detail & Related papers  (2022-08-29T08:47:49Z)
• CLIPstyler: Image Style Transfer with a Single Text Condition [34.24876359759408]
  Existing neural style transfer methods require reference style images to transfer texture information of style images to content images. We propose a new framework that enables a style transfer without' a style image, but only with a text description of the desired style.
  arXiv  Detail & Related papers  (2021-12-01T09:48:53Z)
• Language-Driven Image Style Transfer [72.36790598245096]
  We introduce a new task -- language-driven image style transfer (textttLDIST) -- to manipulate the style of a content image, guided by a text. The discriminator considers the correlation between language and patches of style images or transferred results to jointly embed style instructions. Experiments show that our CLVA is effective and achieves superb transferred results on textttLDIST.
  arXiv  Detail & Related papers  (2021-06-01T01:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.