RSDiff: Remote Sensing Image Generation from Text Using Diffusion Model
- URL: http://arxiv.org/abs/2309.02455v1
- Date: Sun, 3 Sep 2023 09:34:49 GMT
- Title: RSDiff: Remote Sensing Image Generation from Text Using Diffusion Model
- Authors: Ahmad Sebaq, Mohamed ElHelw
- Abstract summary: We propose an innovative and lightweight approach to generate high-resolution Satellite images purely based on text prompts.
Our results demonstrate that our approach outperforms existing state-of-the-art (SoTA) models in generating satellite images with realistic geographical features, weather conditions, and land structures.
- Score: 1.0334138809056097
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Satellite imagery generation and super-resolution are pivotal tasks in remote
sensing, demanding high-quality, detailed images for accurate analysis and
decision-making. In this paper, we propose an innovative and lightweight
approach that employs two-stage diffusion models to gradually generate
high-resolution Satellite images purely based on text prompts. Our innovative
pipeline comprises two interconnected diffusion models: a Low-Resolution
Generation Diffusion Model (LR-GDM) that generates low-resolution images from
text and a Super-Resolution Diffusion Model (SRDM) conditionally produced. The
LR-GDM effectively synthesizes low-resolution by (computing the correlations of
the text embedding and the image embedding in a shared latent space), capturing
the essential content and layout of the desired scenes. Subsequently, the SRDM
takes the generated low-resolution image and its corresponding text prompts and
efficiently produces the high-resolution counterparts, infusing fine-grained
spatial details and enhancing visual fidelity. Experiments are conducted on the
commonly used dataset, Remote Sensing Image Captioning Dataset (RSICD). Our
results demonstrate that our approach outperforms existing state-of-the-art
(SoTA) models in generating satellite images with realistic geographical
features, weather conditions, and land structures while achieving remarkable
super-resolution results for increased spatial precision.
Related papers
- Towards Realistic Data Generation for Real-World Super-Resolution [79.24617577528593]
RealDGen is an unsupervised learning data generation framework designed for real-world super-resolution.
We develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model.
Experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations.
arXiv Detail & Related papers (2024-06-11T13:34:57Z) - Binarized Diffusion Model for Image Super-Resolution [61.963833405167875]
We introduce a novel binarized diffusion model, BI-DiffSR, for image SR.
For the model structure, we design a UNet architecture optimized for binarization.
We propose the consistent-pixel-downsample (CP-Down) and consistent-pixel-upsample (CP-Up) to maintain dimension consistent.
Comprehensive experiments demonstrate that our BI-DiffSR outperforms existing binarization methods.
arXiv Detail & Related papers (2024-06-09T10:30:25Z) - Semantic Guided Large Scale Factor Remote Sensing Image Super-resolution with Generative Diffusion Prior [13.148815217684277]
Large scale factor super-resolution (SR) algorithms are vital for maximizing the utilization of low-resolution (LR) satellite data captured from orbit.
Existing methods confront challenges in recovering SR images with clear textures and correct ground objects.
We introduce a novel framework, the Semantic Guided Diffusion Model (SGDM), designed for large scale factor remote sensing image super-resolution.
arXiv Detail & Related papers (2024-05-11T16:06:16Z) - Recognition-Guided Diffusion Model for Scene Text Image Super-Resolution [15.391125077873745]
Scene Text Image Super-Resolution (STISR) aims to enhance the resolution and legibility of text within low-resolution (LR) images.
Previous methods predominantly employ discriminative Convolutional Neural Networks (CNNs) augmented with diverse forms of text guidance.
We introduce RGDiffSR, a Recognition-Guided Diffusion model for scene text image Super-Resolution, which exhibits great generative diversity and fidelity even in challenging scenarios.
arXiv Detail & Related papers (2023-11-22T11:10:45Z) - DSR-Diff: Depth Map Super-Resolution with Diffusion Model [38.68563026759223]
We present a novel CDSR paradigm that utilizes a diffusion model within the latent space to generate guidance for depth map super-resolution.
Our proposed method has shown superior performance in extensive experiments when compared to state-of-the-art methods.
arXiv Detail & Related papers (2023-11-16T14:18:10Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Implicit Diffusion Models for Continuous Super-Resolution [65.45848137914592]
This paper introduces an Implicit Diffusion Model (IDM) for high-fidelity continuous image super-resolution.
IDM integrates an implicit neural representation and a denoising diffusion model in a unified end-to-end framework.
The scaling factor regulates the resolution and accordingly modulates the proportion of the LR information and generated features in the final output.
arXiv Detail & Related papers (2023-03-29T07:02:20Z) - Exploiting Digital Surface Models for Inferring Super-Resolution for
Remotely Sensed Images [2.3204178451683264]
This paper introduces a novel approach for forcing an SRR model to output realistic remote sensing images.
Instead of relying on feature-space similarities as a perceptual loss, the model considers pixel-level information inferred from the normalized Digital Surface Model (nDSM) of the image.
Based on visual inspection, the inferred super-resolution images exhibit particularly superior quality.
arXiv Detail & Related papers (2022-05-09T06:02:50Z) - Deep Cyclic Generative Adversarial Residual Convolutional Networks for
Real Image Super-Resolution [20.537597542144916]
We consider a deep cyclic network structure to maintain the domain consistency between the LR and HR data distributions.
We propose the Super-Resolution Residual Cyclic Generative Adversarial Network (SRResCycGAN) by training with a generative adversarial network (GAN) framework for the LR to HR domain translation.
arXiv Detail & Related papers (2020-09-07T11:11:18Z) - Real Image Super Resolution Via Heterogeneous Model Ensemble using
GP-NAS [63.48801313087118]
We propose a new method for image superresolution using deep residual network with dense skip connections.
The proposed method won the first place in all three tracks of the AIM 2020 Real Image Super-Resolution Challenge.
arXiv Detail & Related papers (2020-09-02T22:33:23Z) - Unlimited Resolution Image Generation with R2D2-GANs [69.90258455164513]
We present a novel simulation technique for generating high quality images of any predefined resolution.
This method can be used to synthesize sonar scans of size equivalent to those collected during a full-length mission.
The data produced is continuous, realistically-looking, and can also be generated at least two times faster than the real speed of acquisition.
arXiv Detail & Related papers (2020-03-02T17:49:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.