TextureDiffusion: Target Prompt Disentangled Editing for Various Texture Transfer

• URL: http://arxiv.org/abs/2409.09610v2
• Date: Tue, 14 Jan 2025 09:44:01 GMT
• Title: TextureDiffusion: Target Prompt Disentangled Editing for Various Texture Transfer
• Authors: Zihan Su, Junhao Zhuang, Chun Yuan,
• Abstract summary: We propose TextureDiffusion, a tuning-free image editing method applied to various texture transfer. We show that TextureDiffusion can harmoniously transfer various textures with excellent structure and background preservation.
• Score: 32.53299128227546
• License:
• Abstract: Recently, text-guided image editing has achieved significant success. However, existing methods can only apply simple textures like wood or gold when changing the texture of an object. Complex textures such as cloud or fire pose a challenge. This limitation stems from that the target prompt needs to contain both the input image content and <texture>, restricting the texture representation. In this paper, we propose TextureDiffusion, a tuning-free image editing method applied to various texture transfer. Initially, the target prompt is directly set to "<texture>", making the texture disentangled from the input image content to enhance texture representation. Subsequently, query features in self-attention and features in residual blocks are utilized to preserve the structure of the input image. Finally, to maintain the background, we introduce an edit localization technique which blends the self-attention results and the intermediate latents. Comprehensive experiments demonstrate that TextureDiffusion can harmoniously transfer various textures with excellent structure and background preservation. Code is publicly available at https://github.com/THU-CVML/TextureDiffusion

Related papers

• DragText: Rethinking Text Embedding in Point-based Image Editing [3.4248731707266264]
  Point-based image editing enables accurate and flexible control through content dragging. The role of text embedding during the editing process has not been thoroughly investigated. We propose DragText, which optimize text embedding in conjunction with the dragging process to pair with the modified image embedding.
  arXiv  Detail & Related papers  (2024-07-25T07:57:55Z)
• Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis [61.189479577198846]
  We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt. Our approach fine-tunes a diffusion model on a single texture, and learns to embed that statistical distribution in the output domain of the model. At generation time, our fine-tuned diffusion model is used through a score aggregation strategy to generate output texture images of arbitrary resolution on a single GPU.
  arXiv  Detail & Related papers  (2024-05-13T21:53:09Z)
• TexSliders: Diffusion-Based Texture Editing in CLIP Space [17.449209402077276]
  We analyze existing editing methods and show that they are not directly applicable to textures. We propose a novel approach that instead manipulates CLIP image embeddings to condition the diffusion generation.
  arXiv  Detail & Related papers  (2024-05-01T17:57:21Z)
• Compositional Neural Textures [25.885557234297835]
  This work introduces a fully unsupervised approach for representing textures using a compositional neural model. We represent each texton as a 2D Gaussian function whose spatial support approximates its shape, and an associated feature that encodes its detailed appearance. This approach enables a wide range of applications, including transferring appearance from an image texture to another image, diversifying textures, revealing/modifying texture variations, edit propagation, texture animation, and direct texton manipulation.
  arXiv  Detail & Related papers  (2024-04-18T21:09:34Z)
• FlashTex: Fast Relightable Mesh Texturing with LightControlNet [105.4683880648901]
  We introduce LightControlNet, a new text-to-image model based on the ControlNet architecture. We apply our approach to disentangle material/reflectance in the resulting texture so that the mesh can be properlylit and rendered in any lighting environment.
  arXiv  Detail & Related papers  (2024-02-20T18:59:00Z)
• TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion [64.49276500129092]
  TextureDreamer is an image-guided texture synthesis method. It can transfer relightable textures from a small number of input images to target 3D shapes across arbitrary categories.
  arXiv  Detail & Related papers  (2024-01-17T18:55:49Z)
• TEXTure: Text-Guided Texturing of 3D Shapes [71.13116133846084]
  We present TEXTure, a novel method for text-guided editing, editing, and transfer of textures for 3D shapes. We define a trimap partitioning process that generates seamless 3D textures without requiring explicit surface textures.
  arXiv  Detail & Related papers  (2023-02-03T13:18:45Z)
• Exploring Stroke-Level Modifications for Scene Text Editing [86.33216648792964]
  Scene text editing (STE) aims to replace text with the desired one while preserving background and styles of the original text. Previous methods of editing the whole image have to learn different translation rules of background and text regions simultaneously. We propose a novel network by MOdifying Scene Text image at strokE Level (MOSTEL)
  arXiv  Detail & Related papers  (2022-12-05T02:10:59Z)
• DiffEdit: Diffusion-based semantic image editing with mask guidance [64.555930158319]
  DiffEdit is a method to take advantage of text-conditioned diffusion models for the task of semantic image editing. Our main contribution is able to automatically generate a mask highlighting regions of the input image that need to be edited.
  arXiv  Detail & Related papers  (2022-10-20T17:16:37Z)
• Text2LIVE: Text-Driven Layered Image and Video Editing [13.134513605107808]
  We present a method for zero-shot, text-driven appearance manipulation in natural images and videos. Given an input image or video and a target text prompt, our goal is to edit the appearance of existing objects. We demonstrate localized, semantic edits on high-resolution natural images and videos across a variety of objects and scenes.
  arXiv  Detail & Related papers  (2022-04-05T21:17:34Z)
• Texture Transform Attention for Realistic Image Inpainting [6.275013056564918]
  We propose a Texture Transform Attention network that better produces the missing region inpainting with fine details. Texture Transform Attention is used to create a new reassembled texture map using fine textures and coarse semantics. We evaluate our model end-to-end with the publicly available datasets CelebA-HQ and Places2.
  arXiv  Detail & Related papers  (2020-12-08T06:28:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.