Delta-K: Boosting Multi-Instance Generation via Cross-Attention Augmentation

• URL: http://arxiv.org/abs/2603.10210v1
• Date: Tue, 10 Mar 2026 20:23:00 GMT
• Title: Delta-K: Boosting Multi-Instance Generation via Cross-Attention Augmentation
• Abstract summary: Diffusion Models often suffer from concept omission when synthesizing complex scenes.<n>We propose Delta-K, a backbone-agnostic and plug-and-play inference framework that tackles omission by operating directly in the shared cross-attention Key space.
• Score: 3.27529630826945
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Diffusion Models excel in text-to-image synthesis, they often suffer from concept omission when synthesizing complex multi-instance scenes. Existing training-free methods attempt to resolve this by rescaling attention maps, which merely exacerbates unstructured noise without establishing coherent semantic representations. To address this, we propose Delta-K, a backbone-agnostic and plug-and-play inference framework that tackles omission by operating directly in the shared cross-attention Key space. Specifically, with Vision-language model, we extract a differential key $ΔK$ that encodes the semantic signature of missing concepts. This signal is then injected during the early semantic planning stage of the diffusion process. Governed by a dynamically optimized scheduling mechanism, Delta-K grounds diffuse noise into stable structural anchors while preserving existing concepts. Extensive experiments demonstrate the generality of our approach: Delta-K consistently improves compositional alignment across both modern DiT models and classical U-Net architectures, without requiring spatial masks, additional training, or architectural modifications.

Related papers

• Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution [80.28905986756685]
  DiMOO-SR is a rarity-aware multimodal discrete diffusion framework for photo-realistic image Super-Resolution.<n>Experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps.
  arXiv  Detail & Related papers  (2026-07-20T07:01:01Z)
• Discrete Diffusion Models: A Unified Framework from Tokenization to Generation [66.16696467962375]
  This work introduces a unified conceptual framework that views discrete diffusion models through the construction of the underlying discrete state space.<n>Within this framework, existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space.
  arXiv  Detail & Related papers  (2026-07-15T04:23:22Z)
• Not all tokens contribute equally to diffusion learning [16.928860227988086]
  Conditional diffusion models often neglect semantically important tokens during inference, leading to biased or incomplete generations.<n>We propose Distribution-Aware Rectification and Ensemble Spatial Alignment (DARE), a unified framework that improves semantic guidance from perspectives of distributional debiasing and consistency.<n>DARE consistently improves semantic alignment, achieving significant gains over existing approaches.
  arXiv  Detail & Related papers  (2026-04-08T12:45:21Z)
• UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations [45.861324782424326]
  Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap.<n>We introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation.
  arXiv  Detail & Related papers  (2026-03-11T12:14:26Z)
• Forgetting is Competition: Rethinking Unlearning as Representation Interference in Diffusion Models [7.17300076441681]
  SurgUn is a surgical unlearning method that applies targeted weight-space updates to remove specific visual concepts in text-conditioned diffusion models.<n>Our approach is motivated by retroactive interference theory, which holds that newly acquired memories can overwrite, suppress, or impede access to prior ones.<n>We adapt this principle to diffusion models by inducing retroactive concept interference, enabling focused destabilization of only the target concept.
  arXiv  Detail & Related papers  (2026-03-01T08:07:14Z)
• Rethinking Transferable Adversarial Attacks on Point Clouds from a Compact Subspace Perspective [55.919842734983156]
  CoSA is a transferable attack framework that operates within a shared low-dimensional semantic space.<n>CoSA consistently outperforms state-of-the-art transferable attacks.
  arXiv  Detail & Related papers  (2026-01-30T15:48:11Z)
• Bridging the Discrete-Continuous Gap: Unified Multimodal Generation via Coupled Manifold Discrete Absorbing Diffusion [60.186310080523135]
  Bifurcation of generative modeling into autoregressive approaches for discrete data (text) and diffusion approaches for continuous data (images) hinders development of truly unified multimodal systems.<n>We propose textbfCoM-DAD, a novel probabilistic framework that reformulates multimodal generation as a hierarchical dual-process.<n>Our method demonstrates superior stability over standard masked modeling, establishing a new paradigm for scalable, unified text-image generation.
  arXiv  Detail & Related papers  (2026-01-07T16:21:19Z)
• Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge [16.958159611661813]
  Latent Denoising Diffusion Bridge Model (LDDBM) is a general-purpose framework for modality translation.<n>By operating in a shared latent space, our method learns a bridge between arbitrary modalities without requiring aligned dimensions.<n>Our approach supports arbitrary modality pairs and performs strongly on diverse MT tasks, including multi-view to 3D shape generation, image super-resolution, and multi-view scene synthesis.
  arXiv  Detail & Related papers  (2025-10-23T17:59:54Z)
• Generalizing WiFi Gesture Recognition via Large-Model-Aware Semantic Distillation and Alignment [6.124050993047708]
  WiFi-based gesture recognition has emerged as a promising RF sensing paradigm for AIoT environments.<n>We propose a novel generalization framework, termed Large-Model-Aware Semantic Distillation and Alignment.<n>Our method offers a scalable and deployable solution for generalized RF-based gesture interfaces in real-world AIoT applications.
  arXiv  Detail & Related papers  (2025-10-15T10:28:50Z)
• Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Acceleration [24.85624444212476]
  This study proposes a novel concept: attend to prune feature redundancies in areas not attended by the diffusion process.<n>We analyze the location and degree of feature redundancies based on the structure-then-detail denoising priors.<n>We introduce SDTM, a structure-then-detail token merging approach that dynamically compresses feature redundancies.
  arXiv  Detail & Related papers  (2025-05-16T21:27:38Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Semantic Image Synthesis via Diffusion Models [174.24523061460704]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks.<n>Recent work on semantic image synthesis mainly follows the de facto GAN-based approaches.<n>We propose a novel framework based on DDPM for semantic image synthesis.
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.