DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Models

• URL: http://arxiv.org/abs/2502.14779v1
• Date: Thu, 20 Feb 2025 18:01:02 GMT
• Title: DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Models
• Authors: Hongji Yang, Wencheng Han, Yucheng Zhou, Jianbing Shen,
• Abstract summary: We introduce DC (Decouple)-ControlNet, a framework for multi-condition image generation. The core idea behind DC-ControlNet is to decouple control conditions, transforming global control into a hierarchical system. For interactions between elements, we introduce the Inter-Element Controller, which accurately handles multi-element interactions.
• Score: 55.42794740244581
• License:
• Abstract: In this paper, we introduce DC (Decouple)-ControlNet, a highly flexible and precisely controllable framework for multi-condition image generation. The core idea behind DC-ControlNet is to decouple control conditions, transforming global control into a hierarchical system that integrates distinct elements, contents, and layouts. This enables users to mix these individual conditions with greater flexibility, leading to more efficient and accurate image generation control. Previous ControlNet-based models rely solely on global conditions, which affect the entire image and lack the ability of element- or region-specific control. This limitation reduces flexibility and can cause condition misunderstandings in multi-conditional image generation. To address these challenges, we propose both intra-element and Inter-element Controllers in DC-ControlNet. The Intra-Element Controller handles different types of control signals within individual elements, accurately describing the content and layout characteristics of the object. For interactions between elements, we introduce the Inter-Element Controller, which accurately handles multi-element interactions and occlusion based on user-defined relationships. Extensive evaluations show that DC-ControlNet significantly outperforms existing ControlNet models and Layout-to-Image generative models in terms of control flexibility and precision in multi-condition control.

Related papers

• DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation [63.63429658282696]
  We propose DynamicControl, which supports dynamic combinations of diverse control signals. We show that DynamicControl is superior to existing methods in terms of controllability, generation quality and composability under various conditional controls.
  arXiv  Detail & Related papers  (2024-12-04T11:54:57Z)
• AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation [24.07613591217345]
  Linguistic control enables effective content creation, but struggles with fine-grained control over image generation. AnyControl develops a novel Multi-Control framework that extracts a unified multi-modal embedding to guide the generation process. This approach enables a holistic understanding of user inputs, and produces high-quality, faithful results under versatile control signals.
  arXiv  Detail & Related papers  (2024-06-27T07:40:59Z)
• FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation [99.4649330193233]
  Controllable text-to-image (T2I) diffusion models generate images conditioned on both text prompts and semantic inputs of other modalities like edge maps. We propose a novel Flexible and Efficient method, FlexEControl, for controllable T2I generation.
  arXiv  Detail & Related papers  (2024-05-08T06:09:11Z)
• Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model [62.51232333352754]
  Ctrl-Adapter adds diverse controls to any image/video diffusion model through the adaptation of pretrained ControlNets. With six diverse U-Net/DiT-based image/video diffusion models, Ctrl-Adapter matches the performance of pretrained ControlNets on COCO.
  arXiv  Detail & Related papers  (2024-04-15T17:45:36Z)
• Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models [82.19740045010435]
  We introduce Uni-ControlNet, a unified framework that allows for the simultaneous utilization of different local controls and global controls. Unlike existing methods, Uni-ControlNet only requires the fine-tuning of two additional adapters upon frozen pre-trained text-to-image diffusion models. Uni-ControlNet demonstrates its superiority over existing methods in terms of controllability, generation quality and composability.
  arXiv  Detail & Related papers  (2023-05-25T17:59:58Z)
• UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild [166.25327094261038]
  We introduce UniControl, a new generative foundation model for controllable condition-to-image (C2I) tasks. UniControl consolidates a wide array of C2I tasks within a singular framework, while still allowing for arbitrary language prompts. trained on nine unique C2I tasks, UniControl demonstrates impressive zero-shot generation abilities.
  arXiv  Detail & Related papers  (2023-05-18T17:41:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.