Layer-wise Instance Binding for Regional and Occlusion Control in Text-to-Image Diffusion Transformers

• URL: http://arxiv.org/abs/2603.05769v1
• Date: Fri, 06 Mar 2026 00:09:49 GMT
• Title: Layer-wise Instance Binding for Regional and Occlusion Control in Text-to-Image Diffusion Transformers
• Abstract summary: Region-instructed layout control in text-to-image generation is highly practical, yet existing methods suffer from limitations.<n>By modeling regional generation as distinct layers and binding them during the generation, we propose LayerBind.
• Score: 49.08465459791972
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Region-instructed layout control in text-to-image generation is highly practical, yet existing methods suffer from limitations: (i) training-based approaches inherit data bias and often degrade image quality, and (ii) current techniques struggle with occlusion order, limiting real-world usability. To address these issues, we propose LayerBind. By modeling regional generation as distinct layers and binding them during the generation, our method enables precise regional and occlusion controllability. Our motivation stems from the observation that spatial layout and occlusion are established at a very early denoising stage, suggesting that rearranging the early latent structure is sufficient to modify the final output. Building on this, we structure the scheme into two phases: instance initialization and subsequent semantic nursing. (1) First, leveraging the contextual sharing mechanism in multimodal joint attention, Layer-wise Instance Initialization creates per-instance branches that attend to their own regions while anchoring to the shared background. At a designated early step, these branches are fused according to the layer order to form a unified latent with a pre-established layout. (2) Then, Layer-wise Semantic Nursing reinforces regional details and maintains the occlusion order via a layer-wise attention enhancement. Specifically, a sequential layered attention path operates alongside the standard global path, with updates composited under a layer-transparency scheduler. LayerBind is training-free and plug-and-play, serving as a regional and occlusion controller across Diffusion Transformers. Beyond generation, it natively supports editable workflows, allowing for flexible modifications like changing instances or rearranging visible orders. Both qualitative and quantitative results demonstrate LayerBind's effectiveness, highlighting its strong potential for creative applications.

Related papers

• Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization [13.554566178862745]
  Text-to-image personalization aims to generate a user-provided subject in novel scenes described by text.<n>Most existing methods encode subject identity (fidelity) and context (editability) through the same conditioning pathway.<n>We propose Decoupled Guidance (DeGu), a framework that routes subject identity and scene context through two independent guidance streams.
  arXiv  Detail & Related papers  (2026-07-01T10:48:43Z)
• Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories [62.54876287800644]
  Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric.<n>Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction.<n>We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable.
  arXiv  Detail & Related papers  (2026-06-21T14:25:50Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• SpectraFlow: Unifying Structural Pretraining and Frequency Adaptation for Medical Image Segmentation [3.182068203318624]
  Recent self-supervised pretraining has improved transferability, but it often exhibits a texture bias.<n>We propose a two-stage framework that couples structure-aware encoder pretraining with boundary-oriented decoding.<n> Experiments on ISIC-2016, Kvasir-SEG, and GlaS demonstrate consistent gains over state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-14T08:37:10Z)
• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues [106.02577891104079]
  We propose MagicQuill V2, a novel system that introduces a textbflayered composition paradigm to generative image editing.<n>Our method deconstructs creative intent into a stack of controllable visual cues.
  arXiv  Detail & Related papers  (2025-12-02T18:59:58Z)
• Rethinking Layer-wise Model Merging through Chain of Merges [21.26982153528304]
  Chain of Merges (CoM) is a layer-wise merging procedure that sequentially merges weights across layers while sequentially updating activation statistics.<n> Experiments on standard benchmarks demonstrate that CoM achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-08-29T08:44:47Z)
• Control and Realism: Best of Both Worlds in Layout-to-Image without Training [59.16447569868382]
  We present WinWinLay, a training-free method for layout-to-Image generation.<n>We propose two key strategies, Non-local Attention Energy and Adaptive Update, that collaboratively enhance control precision and realism.<n>WinWinLay excels in controlling element placement and achieving photorealistic visual fidelity, outperforming the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-06-18T15:39:02Z)
• PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generation [24.964136963713102]
  We present PixelPonder, a novel unified control framework that allows for effective control of multiple visual conditions under a single control structure.<n>Specifically, we design a patch-level adaptive condition selection mechanism that dynamically prioritizes spatially relevant control signals at the sub-region level.<n>Extensive experiments demonstrate that PixelPonder surpasses previous methods across different benchmark datasets.
  arXiv  Detail & Related papers  (2025-03-09T16:27:02Z)
• ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation [108.69315278353932]
  We introduce the Anonymous Region Transformer (ART), which facilitates the direct generation of variable multi-layer transparent images.<n>By enabling precise control and scalable layer generation, ART establishes a new paradigm for interactive content creation.
  arXiv  Detail & Related papers  (2025-02-25T16:57:04Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• Cross-Enhancement Transformer for Action Segmentation [5.752561578852787]
  A novel encoder-decoder structure is proposed in this paper, called Cross-Enhancement Transformer. Our approach can be effective learning of temporal structure representation with interactive self-attention mechanism. In addition, a new loss function is proposed to enhance the training process that penalizes over-segmentation errors.
  arXiv  Detail & Related papers  (2022-05-19T10:06:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.