Zero-Shot Visual Concept Blending Without Text Guidance

• URL: http://arxiv.org/abs/2503.21277v2
• Date: Tue, 01 Apr 2025 06:19:44 GMT
• Title: Zero-Shot Visual Concept Blending Without Text Guidance
• Authors: Hiroya Makino, Takahiro Yamaguchi, Hiroyuki Sakai,
• Abstract summary: "Visual Concept Blending" provides fine-grained control over which features from multiple reference images are transferred to a source image.<n>Our method enables the flexible transfer of texture, shape, motion, style, and more abstract conceptual transformations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel, zero-shot image generation technique called "Visual Concept Blending" that provides fine-grained control over which features from multiple reference images are transferred to a source image. If only a single reference image is available, it is difficult to isolate which specific elements should be transferred. However, using multiple reference images, the proposed approach distinguishes between common and unique features by selectively incorporating them into a generated output. By operating within a partially disentangled Contrastive Language-Image Pre-training (CLIP) embedding space (from IP-Adapter), our method enables the flexible transfer of texture, shape, motion, style, and more abstract conceptual transformations without requiring additional training or text prompts. We demonstrate its effectiveness across a diverse range of tasks, including style transfer, form metamorphosis, and conceptual transformations, showing how subtle or abstract attributes (e.g., brushstroke style, aerodynamic lines, and dynamism) can be seamlessly combined into a new image. In a user study, participants accurately recognized which features were intended to be transferred. Its simplicity, flexibility, and high-level control make Visual Concept Blending valuable for creative fields such as art, design, and content creation, where combining specific visual qualities from multiple inspirations is crucial.

Related papers

• Flux Already Knows -- Activating Subject-Driven Image Generation without Training [25.496237241889048]
  We propose a zero-shot framework for subject-driven image generation using a vanilla Flux model. We activate strong identity-preserving capabilities without any additional data, training, or inference-time fine-tuning.
  arXiv  Detail & Related papers  (2025-04-12T20:41:53Z)
• ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation [108.69315278353932]
  We introduce the Anonymous Region Transformer (ART), which facilitates the direct generation of variable multi-layer transparent images. By enabling precise control and scalable layer generation, ART establishes a new paradigm for interactive content creation.
  arXiv  Detail & Related papers  (2025-02-25T16:57:04Z)
• IP-Composer: Semantic Composition of Visual Concepts [49.18472621931207]
  We present IP-Composer, a training-free approach for compositional image generation.<n>Our method builds on IP-Adapter, which synthesizes novel images conditioned on an input image's CLIP embedding.<n>We extend this approach to multiple visual inputs by crafting composite embeddings, stitched from the projections of multiple input images onto concept-specific CLIP-subspaces identified through text.
  arXiv  Detail & Related papers  (2025-02-19T18:49:31Z)
• ArtCrafter: Text-Image Aligning Style Transfer via Embedding Reframing [22.054292195271476]
  ArtCrafter is a novel framework for text-to-image style transfer.<n>We introduce an attention-based style extraction module.<n>We also present a novel text-image aligning augmentation component.
  arXiv  Detail & Related papers  (2025-01-03T19:17:27Z)
• Object-level Visual Prompts for Compositional Image Generation [75.6085388740087]
  We introduce a method for composing object-level visual prompts within a text-to-image diffusion model.<n>A key challenge in this task is to preserve the identity of the objects depicted in the input visual prompts.<n>We introduce a new KV-mixed cross-attention mechanism, in which keys and values are learned from distinct visual representations.
  arXiv  Detail & Related papers  (2025-01-02T18:59:44Z)
• MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Models [51.1034358143232]
  We introduce component-controllable personalization, a new task that allows users to customize and reconfigure individual components within concepts.<n>This task faces two challenges: semantic pollution, where undesirable elements distort the concept, and semantic imbalance, which leads to disproportionate learning of the target concept and component.<n>We design MagicTailor, a framework that uses Dynamic Masked Degradation to adaptively perturb unwanted visual semantics and Dual-Stream Balancing for more balanced learning of desired visual semantics.
  arXiv  Detail & Related papers  (2024-10-17T09:22:53Z)
• Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization [56.12990759116612]
  Pick-and-Draw is a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods. The proposed approach can be applied to any personalized diffusion models and requires as few as a single reference image.
  arXiv  Detail & Related papers  (2024-01-30T05:56:12Z)
• Decoupled Textual Embeddings for Customized Image Generation [62.98933630971543]
  Customized text-to-image generation aims to learn user-specified concepts with a few images. Existing methods usually suffer from overfitting issues and entangle the subject-unrelated information with the learned concept. We propose the DETEX, a novel approach that learns the disentangled concept embedding for flexible customized text-to-image generation.
  arXiv  Detail & Related papers  (2023-12-19T03:32:10Z)
• Break-A-Scene: Extracting Multiple Concepts from a Single Image [80.47666266017207]
  We introduce the task of textual scene decomposition. We propose augmenting the input image with masks that indicate the presence of target concepts. We then present a novel two-phase customization process.
  arXiv  Detail & Related papers  (2023-05-25T17:59:04Z)
• Highly Personalized Text Embedding for Image Manipulation by Stable Diffusion [34.662798793560995]
  We present a simple yet highly effective approach to personalization using highly personalized (PerHi) text embedding. Our method does not require model fine-tuning or identifiers, yet still enables manipulation of background, texture, and motion with just a single image and target text.
  arXiv  Detail & Related papers  (2023-03-15T17:07:45Z)
• Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation [10.39028769374367]
  We present a new framework that takes text-to-image synthesis to the realm of image-to-image translation. Our method harnesses the power of a pre-trained text-to-image diffusion model to generate a new image that complies with the target text.
  arXiv  Detail & Related papers  (2022-11-22T20:39:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.