FreeBlend: Advancing Concept Blending with Staged Feedback-Driven Interpolation Diffusion

• URL: http://arxiv.org/abs/2502.05606v3
• Date: Sun, 09 Nov 2025 11:33:47 GMT
• Title: FreeBlend: Advancing Concept Blending with Staged Feedback-Driven Interpolation Diffusion
• Abstract summary: Concept blending is a promising yet underexplored area in generative models.<n>We introduce FreeBlend, an effective, training-free framework designed to address these challenges.
• Score: 9.51081615960585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Concept blending is a promising yet underexplored area in generative models. While recent approaches, such as embedding mixing and latent modification based on structural sketches, have been proposed, they often suffer from incompatible semantic information and discrepancies in shape and appearance. In this work, we introduce FreeBlend, an effective, training-free framework designed to address these challenges. To mitigate cross-modal loss and enhance feature detail, we leverage transferred image embeddings as conditional inputs. The framework employs a stepwise increasing interpolation strategy between latents, progressively adjusting the blending ratio to seamlessly integrate auxiliary features. Additionally, we introduce a feedback-driven mechanism that updates the auxiliary latents in reverse order, facilitating global blending and preventing rigid or unnatural outputs. Extensive experiments demonstrate that our method significantly improves both the semantic coherence and visual quality of blended images, yielding compelling and coherent results.

Related papers

• Isotropic Embedding Perturbations for Robust Vision Language Encoders [73.92626712781859]
  Aether is a simple plug-in method that applies diffusion-style random perturbations in the embedding space via controlled alpha-mixing.<n>Inspired by feature-space perturbations in language models and image degradation in generative pretraining, Aether induces mild yet effective perturbations.<n>Aether delivers consistent gains over the advanced recipe combining CutMix, Mixup, DropPath, and RandAug.
  arXiv  Detail & Related papers  (2026-09-09T15:08:05Z)
• Coevolving Representations in Joint Image-Feature Diffusion [13.292849179066993]
  We argue that the representation space guiding diffusion should itself adapt to the generative task.<n>We propose Coevolving Diffusion Representation (CoReDi), a framework in which the semantic representation space evolves during training.<n>Experiments show that CoReDi achieves faster convergence and higher sample quality compared to joint diffusion models operating in fixed representation spaces.
  arXiv  Detail & Related papers  (2026-04-19T15:29:15Z)
• Adaptive Auxiliary Prompt Blending for Target-Faithful Diffusion Generation [6.826122099204317]
  We introduce Adaptive Auxiliary Prompt Blending (AAPB) to stabilize the diffusion process in low-density regions.<n>AAPB provides semantic support in rare concept generation and structural support in image editing.<n>We empirically show consistent improvements on the RareBench and FlowEdit datasets.
  arXiv  Detail & Related papers  (2026-03-19T17:12:03Z)
• Cycle-Consistent Tuning for Layered Image Decomposition [26.331480224165364]
  Disentangling visual layers in real-world images is a persistent challenge in vision and graphics.<n>We present an in-context image decomposition framework that leverages large diffusion foundation models for layered separation.<n>Our approach achieves accurate and coherent decompositions and also generalizes effectively across other decomposition types.
  arXiv  Detail & Related papers  (2026-02-24T15:10:31Z)
• Reversible Efficient Diffusion for Image Fusion [66.35113261837469]
  Multi-modal image fusion aims to consolidate complementary information from diverse source images into a unified representation.<n>While diffusion models have demonstrated impressive generative capabilities in image generation, they often suffer from detail loss when applied to image fusion tasks.<n>This issue arises from the accumulation of noise errors inherent in the Markov process, leading to inconsistency and degradation in the fused results.<n>We propose the Reversible Efficient Diffusion (RED) model - an explicitly supervised training framework that inherits the powerful generative capability of diffusion models while avoiding the distribution estimation.
  arXiv  Detail & Related papers  (2026-01-28T05:14:55Z)
• Sissi: Zero-shot Style-guided Image Synthesis via Semantic-style Integration [57.02757226679549]
  We introduce a training-free framework that reformulates style-guided synthesis as an in-context learning task.<n>We propose a Dynamic Semantic-Style Integration (DSSI) mechanism that reweights attention between semantic and style visual tokens.<n>Experiments show that our approach achieves high-fidelity stylization with superior semantic-style balance and visual quality.
  arXiv  Detail & Related papers  (2026-01-10T16:01:14Z)
• StableSketcher: Enhancing Diffusion Model for Pixel-based Sketch Generation via Visual Question Answering Feedback [4.851573895718146]
  We propose StableSketcher, a novel framework that empowers diffusion models to generate hand-drawn sketches with high prompt fidelity.<n>We fine-tune the variational autoencoder to optimize latent decoding, enabling it to better capture the characteristics of sketches.<n>In parallel, we integrate a new reward function for reinforcement learning based on visual question answering, which improves text-image alignment and semantic consistency.
  arXiv  Detail & Related papers  (2025-10-23T00:27:32Z)
• From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusion [98.31811240195324]
  ConFill is a novel framework that reduces discrepancies between generated and original images at each diffusion step. It outperforms current methods, setting a new benchmark in image completion.
  arXiv  Detail & Related papers  (2025-04-19T13:40:46Z)
• Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method [60.88467353578118]
  We show that a fixed-point-inspired iterative approach to invert real-world images does not achieve convergence, instead oscillating between distinct clusters. We introduce a simple and fast distribution transfer technique that facilitates image enhancement, stroke-based recoloring, as well as visual prompt-guided image editing.
  arXiv  Detail & Related papers  (2024-11-17T17:45:37Z)
• Merging and Splitting Diffusion Paths for Semantically Coherent Panoramas [33.334956022229846]
  We propose the Merge-Attend-Diffuse operator, which can be plugged into different types of pretrained diffusion models used in a joint diffusion setting. Specifically, we merge the diffusion paths, reprogramming self- and cross-attention to operate on the aggregated latent space. Our method maintains compatibility with the input prompt and visual quality of the generated images while increasing their semantic coherence.
  arXiv  Detail & Related papers  (2024-08-28T09:22:32Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• Training-free Composite Scene Generation for Layout-to-Image Synthesis [29.186425845897947]
  This paper introduces a novel training-free approach designed to overcome adversarial semantic intersections during the diffusion conditioning phase. We propose two innovative constraints: 1) an inter-token constraint that resolves token conflicts to ensure accurate concept synthesis; and 2) a self-attention constraint that improves pixel-to-pixel relationships. Our evaluations confirm the effectiveness of leveraging layout information for guiding the diffusion process, generating content-rich images with enhanced fidelity and complexity.
  arXiv  Detail & Related papers  (2024-07-18T15:48:07Z)
• Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution [80.85121353651554]
  We introduce kernel-wise differential operations within the convolutional kernel and develop several learnable directional gradient convolutions. These convolutions are integrated in parallel with a novel linear weighting mechanism to form an Adaptive Directional Gradient Convolution (DGConv) We further devise an Adaptive Information Interaction Block (AIIBlock) to adeptly balance the enhancement of texture and contrast while meticulously investigating the interdependencies, culminating in the creation of a DGPNet for Real-SR through simple stacking.
  arXiv  Detail & Related papers  (2024-05-11T14:21:40Z)
• Training-Free Semantic Video Composition via Pre-trained Diffusion Model [96.0168609879295]
  Current approaches, predominantly trained on videos with adjusted foreground color and lighting, struggle to address deep semantic disparities beyond superficial adjustments. We propose a training-free pipeline employing a pre-trained diffusion model imbued with semantic prior knowledge. Experimental results reveal that our pipeline successfully ensures the visual harmony and inter-frame coherence of the outputs.
  arXiv  Detail & Related papers  (2024-01-17T13:07:22Z)
• Compositional Inversion for Stable Diffusion Models [64.79261401944994]
  Inversion methods generate personalized images by incorporating concepts of interest provided by user images. Existing methods often suffer from overfitting issues, where the dominant presence of inverted concepts leads to the absence of other desired concepts. We propose a method that guides the inversion process towards the core distribution for compositional embeddings.
  arXiv  Detail & Related papers  (2023-12-13T10:57:46Z)
• Fine-grained Appearance Transfer with Diffusion Models [23.29713777525402]
  Image-to-image translation (I2I) seeks to alter the visual appearance between images while maintaining structural coherence. This paper proposes an innovative framework designed to surmount these challenges by integrating various aspects of semantic matching, appearance transfer, and latent deviation.
  arXiv  Detail & Related papers  (2023-11-27T04:00:04Z)
• Contextual Fusion For Adversarial Robustness [0.0]
  Deep neural networks are usually designed to process one particular information stream and susceptible to various types of adversarial perturbations. We developed a fusion model using a combination of background and foreground features extracted in parallel from Places-CNN and Imagenet-CNN. For gradient based attacks, our results show that fusion allows for significant improvements in classification without decreasing performance on unperturbed data.
  arXiv  Detail & Related papers  (2020-11-18T20:13:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.