ObjectAdd: Adding Objects into Image via a Training-Free Diffusion Modification Fashion

• URL: http://arxiv.org/abs/2404.17230v2
• Date: Thu, 2 May 2024 14:57:37 GMT
• Title: ObjectAdd: Adding Objects into Image via a Training-Free Diffusion Modification Fashion
• Authors: Ziyue Zhang, Mingbao Lin, Rongrong Ji,
• Abstract summary: We introduce ObjectAdd, a training-free diffusion modification method to add user-expected objects into user-specified area. With a text-prompted image, our ObjectAdd allows users to specify a box and an object, and achieves: (1) adding object inside the box area; (2) exact content outside the box area; (3) flawless fusion between the two areas.
• Score: 68.3013463352728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce ObjectAdd, a training-free diffusion modification method to add user-expected objects into user-specified area. The motive of ObjectAdd stems from: first, describing everything in one prompt can be difficult, and second, users often need to add objects into the generated image. To accommodate with real world, our ObjectAdd maintains accurate image consistency after adding objects with technical innovations in: (1) embedding-level concatenation to ensure correct text embedding coalesce; (2) object-driven layout control with latent and attention injection to ensure objects accessing user-specified area; (3) prompted image inpainting in an attention refocusing & object expansion fashion to ensure rest of the image stays the same. With a text-prompted image, our ObjectAdd allows users to specify a box and an object, and achieves: (1) adding object inside the box area; (2) exact content outside the box area; (3) flawless fusion between the two areas

Related papers

• DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting [63.01425442236011]
  We present DreamMix, a diffusion-based generative model adept at inserting target objects into scenes at user-specified locations. We propose an Attribute Decoupling Mechanism (ADM) and a Textual Attribute Substitution (TAS) module to improve the diversity and discriminative capability of the text-based attribute guidance.
  arXiv  Detail & Related papers  (2024-11-26T08:44:47Z)
• Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models [78.90023746996302]
  Add-it is a training-free approach that extends diffusion models' attention mechanisms to incorporate information from three key sources. Our weighted extended-attention mechanism maintains structural consistency and fine details while ensuring natural object placement. Human evaluations show that Add-it is preferred in over 80% of cases.
  arXiv  Detail & Related papers  (2024-11-11T18:50:09Z)
• Improving Text-guided Object Inpainting with Semantic Pre-inpainting [95.17396565347936]
  We decompose the typical single-stage object inpainting into two cascaded processes: semantic pre-inpainting and high-fieldity object generation. To achieve this, we cascade a Transformer-based semantic inpainter and an object inpainting diffusion model, leading to a novel CAscaded Transformer-Diffusion framework.
  arXiv  Detail & Related papers  (2024-09-12T17:55:37Z)
• Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model [81.96954332787655]
  We introduce Diffree, a Text-to-Image (T2I) model that facilitates text-guided object addition with only text control. In experiments, Diffree adds new objects with a high success rate while maintaining background consistency, spatial, and object relevance and quality.
  arXiv  Detail & Related papers  (2024-07-24T03:58:58Z)
• Customizing Text-to-Image Diffusion with Camera Viewpoint Control [53.621518249820745]
  We introduce a new task -- enabling explicit control of camera viewpoint for model customization. This allows us to modify object properties amongst various background scenes via text prompts. We propose to condition the 2D diffusion process on rendered, view-dependent features of the new object.
  arXiv  Detail & Related papers  (2024-04-18T16:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.