Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization

• URL: http://arxiv.org/abs/2607.00766v2
• Date: Thu, 02 Jul 2026 03:21:52 GMT
• Title: Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization
• Abstract summary: Text-to-image personalization aims to generate a user-provided subject in novel scenes described by text.<n>Most existing methods encode subject identity (fidelity) and context (editability) through the same conditioning pathway.<n>We propose Decoupled Guidance (DeGu), a framework that routes subject identity and scene context through two independent guidance streams.
• Score: 13.554566178862745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-image personalization aims to generate a user-provided subject in novel scenes described by text. However, most existing methods encode subject identity (fidelity) and context (editability) through the same conditioning pathway, forcing the two to compete for attention-map resources. We refer to this phenomenon as conditioning entanglement and show that it induces a fidelity-editability trade-off. We further provide causal evidence by replacing the target subject token with a generic subject token, which produces shifts in attention allocation and corresponding changes in context adherence. To this end, we propose Decoupled Guidance (DeGu), a plug-and-play framework that routes subject identity and scene context through two independent guidance streams. We further introduce a spatial mixing mechanism that dynamically fuses these streams, ensuring each operates within its semantically relevant region without interference. Furthermore, DeGu can be readily applied to existing personalization methods without modifying the underlying backbone models, consistently improving the overall personalization performance while enabling inference-time control over the fidelity-editability balance, across diverse methods and backbones, including flow-matching Diffusion Transformers (DiTs).

Related papers

• Redirecting the Flow: Image Customization through Attention Distribution Shift [18.601789249339014]
  Subject-driven image customization aims to generate images that follow textual instructions and preserve the identity of a given reference subject.<n>Existing approaches suffer from limited efficiency, misalignment between extracted reference features and the generative process, and interference from irrelevant information.<n>We propose CustomShift, a dual-branch architecture based on Stable Diffusion 3.<n> Experiments on the DreamBooth and Custom101 benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-06-15T15:44:14Z)
• Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework [52.477407591581176]
  Mask-free video object insertion has emerged as a challenging task, requiring harmonious integration of reference objects into source videos.<n>We propose textittextbfSmart-Insertion-V, an end-to-end textbfDual-Stream framework that concurrently conducts video insertion and image style transfer.
  arXiv  Detail & Related papers  (2026-05-22T17:54:54Z)
• Semantic-Structural Alignment for Generative Pictorial Charts [59.45629259524998]
  We present a generative framework for the automated synthesis of pictorial charts.<n>We frame the problem as a dual-conditioned generation task guided by two parallel external control signals.<n>Our method produces pictorial charts that are both artistically compelling and structurally consistent.
  arXiv  Detail & Related papers  (2026-05-05T05:20:46Z)
• A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargeting [26.68081874066983]
  We present a unifying perspective where both tasks are cast as instances of conditional transport within a single generative framework.<n>We implement this vision via a rectified-flow motion model jointly conditioned and target skeletal structures.<n>Experiments on SnapMoGen and a multi-character Mixamo subset show that a single trained model supports text-to-motion generation, zero-shot editing, and zero-shot intra-structural.
  arXiv  Detail & Related papers  (2026-04-15T02:53:07Z)
• Layer-wise Instance Binding for Regional and Occlusion Control in Text-to-Image Diffusion Transformers [49.08465459791972]
  Region-instructed layout control in text-to-image generation is highly practical, yet existing methods suffer from limitations.<n>By modeling regional generation as distinct layers and binding them during the generation, we propose LayerBind.
  arXiv  Detail & Related papers  (2026-03-06T00:09:49Z)
• Shifting the Breaking Point of Flow Matching for Multi-Instance Editing [47.32746672482526]
  We introduce Instance-Disentangled Attention, a mechanism that partitions joint attention operations and enforces binding between instance-specific textual instructions and spatial regions.<n>Our approach promotes edit disentanglement and locality while preserving global output coherence, enabling single-pass, instance-level editing.
  arXiv  Detail & Related papers  (2026-02-09T14:52:45Z)
• Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model [60.82962950960996]
  We introduce UnifyEdit, a tuning-free method that performs diffusion latent optimization.<n>We develop two attention-based constraints: a self-attention (SA) preservation constraint for structural fidelity, and a cross-attention (CA) alignment constraint to enhance text alignment.<n>Our approach achieves a robust balance between structure preservation and text alignment across various editing tasks, outperforming other state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-08T01:02:50Z)
• Unlocking the Potential of Text-to-Image Diffusion with PAC-Bayesian Theory [33.78620829249978]
  Text-to-image (T2I) diffusion models have revolutionized generative modeling by producing high-fidelity, diverse, and visually realistic images. Recent attention-based methods have improved object inclusion and linguistic binding, but still face challenges such as attribute misbinding. We propose a Bayesian approach that designs custom priors over attention distributions to enforce desirable properties. Our approach treats the attention mechanism as an interpretable component, enabling fine-grained control and improved attribute-object alignment.
  arXiv  Detail & Related papers  (2024-11-25T10:57:48Z)
• Learning to Manipulate Individual Objects in an Image [71.55005356240761]
  We describe a method to train a generative model with latent factors that are independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects. Unlike other unsupervised generative models, ours enables object-centric manipulation, without requiring object-level annotations.
  arXiv  Detail & Related papers  (2020-04-11T21:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.