CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model

• URL: http://arxiv.org/abs/2403.11614v4
• Date: Sun, 1 Sep 2024 06:32:06 GMT
• Title: CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model
• Authors: Datao Tang, Xiangyong Cao, Xingsong Hou, Zhongyuan Jiang, Junmin Liu, Deyu Meng,
• Abstract summary: CRS-Diff is a new RS generative framework specifically tailored for RS image generation. To our knowledge, CRS-Diff is the first multiple-condition controllable RS generative model. Our CRS-Diff can serve as a data engine that generates high-quality training data for downstream tasks.
• Score: 42.92146478120197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of generative models has revolutionized the field of remote sensing (RS) image generation. Despite generating high-quality images, existing methods are limited in relying mainly on text control conditions, and thus do not always generate images accurately and stably. In this paper, we propose CRS-Diff, a new RS generative framework specifically tailored for RS image generation, leveraging the inherent advantages of diffusion models while integrating more advanced control mechanisms. Specifically, CRS-Diff can simultaneously support text-condition, metadata-condition, and image-condition control inputs, thus enabling more precise control to refine the generation process. To effectively integrate multiple condition control information, we introduce a new conditional control mechanism to achieve multi-scale feature fusion, thus enhancing the guiding effect of control conditions. To our knowledge, CRS-Diff is the first multiple-condition controllable RS generative model. Experimental results in single-condition and multiple-condition cases have demonstrated the superior ability of our CRS-Diff to generate RS images both quantitatively and qualitatively compared with previous methods. Additionally, our CRS-Diff can serve as a data engine that generates high-quality training data for downstream tasks, e.g., road extraction. The code is available at https://github.com/Sonettoo/CRS-Diff.

Related papers

• ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention [86.93601565563954]
  ScaleWeaver is a framework designed to achieve high-fidelity, controllable generation upon advanced visual autoregressive( VAR) models.<n>The proposed Reference Attention module discards the unnecessary attention from image$rightarrow$condition, reducing computational cost.<n>Experiments show that ScaleWeaver delivers high-quality generation and precise control while attaining superior efficiency over diffusion-based methods.
  arXiv  Detail & Related papers  (2025-10-16T17:00:59Z)
• SCALAR: Scale-wise Controllable Visual Autoregressive Learning [15.775596699630633]
  We present SCALAR, a controllable generation method based on Visual Autoregressive ( VAR)<n>We leverage a pretrained image encoder to extract semantic control signal encodings, which are projected into scale-specific representations and injected into the corresponding layers of the VAR backbone.<n>Building on SCALAR, we develop SCALAR-Uni, a unified extension that aligns multiple control modalities into a shared latent space, supporting flexible multi-conditional guidance in a single model.
  arXiv  Detail & Related papers  (2025-07-26T13:23:08Z)
• Enhancing Remote Sensing Vision-Language Models Through MLLM and LLM-Based High-Quality Image-Text Dataset Generation [21.62138893025555]
  A key challenge lies in the scarcity of high-quality, large-scale, image-text paired training data.<n>We propose a two-stage method named MpGI for generating high-quality text captions for RS images.<n>We fine-tuned two VLFMs using our dataset: CLIP, a discriminative model, and CoCa, an image-to-text generative model.
  arXiv  Detail & Related papers  (2025-07-22T15:54:53Z)
• DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation [63.63429658282696]
  We propose DynamicControl, which supports dynamic combinations of diverse control signals. We show that DynamicControl is superior to existing methods in terms of controllability, generation quality and composability under various conditional controls.
  arXiv  Detail & Related papers  (2024-12-04T11:54:57Z)
• OminiControl: Minimal and Universal Control for Diffusion Transformer [68.3243031301164]
  OminiControl is a framework that integrates image conditions into pre-trained Diffusion Transformer (DiT) models. At its core, OminiControl leverages a parameter reuse mechanism, enabling the DiT to encode image conditions using itself as a powerful backbone. OminiControl addresses a wide range of image conditioning tasks in a unified manner, including subject-driven generation and spatially-aligned conditions.
  arXiv  Detail & Related papers  (2024-11-22T17:55:15Z)
• ControlSR: Taming Diffusion Models for Consistent Real-World Image Super Resolution [68.72454974431749]
  We present ControlSR, a new method that can tame Diffusion Models for consistent real-world image super-resolution (Real-ISR) Our model can achieve better performance across multiple metrics on several test sets and generate more consistent SR results with LR images than existing methods.
  arXiv  Detail & Related papers  (2024-10-18T08:35:57Z)
• A Simple Approach to Unifying Diffusion-based Conditional Generation [63.389616350290595]
  We introduce a simple, unified framework to handle diverse conditional generation tasks. Our approach enables versatile capabilities via different inference-time sampling schemes. Our model supports additional capabilities like non-spatially aligned and coarse conditioning.
  arXiv  Detail & Related papers  (2024-10-15T09:41:43Z)
• CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation [69.43106794519193]
  We propose the CtrLoRA framework, which trains a Base ControlNet to learn the common knowledge of image-to-image generation from multiple base conditions. Our framework reduces the learnable parameters by 90% compared to ControlNet, significantly lowering the threshold to distribute and deploy the model weights.
  arXiv  Detail & Related papers  (2024-10-12T07:04:32Z)
• ControlAR: Controllable Image Generation with Autoregressive Models [40.74890550081335]
  We introduce ControlAR, an efficient framework for integrating spatial controls into autoregressive image generation models. ControlAR exploits the conditional decoding method to generate the next image token conditioned on the per-token fusion between control and image tokens. Results indicate that ControlAR surpasses previous state-of-the-art controllable diffusion models.
  arXiv  Detail & Related papers  (2024-10-03T17:28:07Z)
• SelfDRSC++: Self-Supervised Learning for Dual Reversed Rolling Shutter Correction [72.05587640928879]
  We propose an enhanced Self-supervised learning framework for Dual reversed RS distortion Correction (SelfDRSC++) We introduce a lightweight DRSC network that incorporates a bidirectional correlation matching block to refine the joint optimization of optical flows and corrected RS features. To effectively train the DRSC network, we propose a self-supervised learning strategy that ensures cycle consistency between input and reconstructed dual reversed RS images.
  arXiv  Detail & Related papers  (2024-08-21T08:17:22Z)
• ControlVAR: Exploring Controllable Visual Autoregressive Modeling [48.66209303617063]
  Conditional visual generation has witnessed remarkable progress with the advent of diffusion models (DMs) Challenges such as expensive computational cost, high inference latency, and difficulties of integration with large language models (LLMs) have necessitated exploring alternatives to DMs. This paper introduces Controlmore, a novel framework that explores pixel-level controls in visual autoregressive modeling for flexible and efficient conditional generation.
  arXiv  Detail & Related papers  (2024-06-14T06:35:33Z)
• Intriguing Property and Counterfactual Explanation of GAN for Remote Sensing Image Generation [25.96740500337747]
  Generative adversarial networks (GANs) have achieved remarkable progress in the natural image field. GAN model is more sensitive to the size of training data for RS image generation than for natural image generation. We propose two innovative adjustment schemes, namely Uniformity Regularization (UR) and Entropy Regularization (ER), to increase the information learned by the GAN model.
  arXiv  Detail & Related papers  (2023-03-09T13:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.