Non-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery

• URL: http://arxiv.org/abs/2311.17486v1
• Date: Wed, 29 Nov 2023 09:48:01 GMT
• Title: Non-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery
• Authors: Zichen Tian, Zhaozheng Chen, Qianru Sun
• Abstract summary: We explore the "hidden" ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains. We propose a 2-stage low-rank adaptation method, and we call it 2LoRA. In the first stage, the model is adapted using aerial-view regular image data (whose structure matches SAR), followed by the second stage where the base model from the first stage is further adapted using SAR modality data.
• Score: 30.590315753622132
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We explore the "hidden" ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains, taking Synthetic Aperture Radar (SAR) data for a case study. Due to the inherent challenges in capturing satellite data, acquiring ample SAR training samples is infeasible. For instance, for a particular category of ship in the open sea, we can collect only few-shot SAR images which are too limited to derive effective ship recognition models. If large-scale models pre-trained with regular images can be adapted to generating novel SAR images, the problem is solved. In preliminary study, we found that fine-tuning these models with few-shot SAR images is not working, as the models can not capture the two primary differences between SAR and regular images: structure and modality. To address this, we propose a 2-stage low-rank adaptation method, and we call it 2LoRA. In the first stage, the model is adapted using aerial-view regular image data (whose structure matches SAR), followed by the second stage where the base model from the first stage is further adapted using SAR modality data. Particularly in the second stage, we introduce a novel prototype LoRA (pLoRA), as an improved version of 2LoRA, to resolve the class imbalance problem in SAR datasets. For evaluation, we employ the resulting generation model to synthesize additional SAR data. This augmentation, when integrated into the training process of SAR classification as well as segmentation models, yields notably improved performance for minor classes

Related papers

• Rethinking Image Super-Resolution from Training Data Perspectives [54.28824316574355]
  We investigate the understudied effect of the training data used for image super-resolution (SR) With this, we propose an automated image evaluation pipeline. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance.
  arXiv  Detail & Related papers  (2024-09-01T16:25:04Z)
• Conditional Brownian Bridge Diffusion Model for VHR SAR to Optical Image Translation [5.578820789388206]
  This paper introduces a conditional image-to-image translation approach based on Brownian Bridge Diffusion Model (BBDM) We conducted comprehensive experiments on the MSAW dataset, a paired SAR and optical images collection of 0.5m Very-High-Resolution (VHR)
  arXiv  Detail & Related papers  (2024-08-15T05:43:46Z)
• SARatrX: Towards Building A Foundation Model for SAR Target Recognition [22.770010893572973]
  We make the first attempt towards building a foundation model for SAR ATR, termed SARatrX. SARatrX learns generalizable representations via self-supervised learning (SSL) and provides a basis for label-efficient model adaptation to generic SAR target detection and classification tasks. Specifically, SARatrX is trained on 0.18 M unlabelled SAR target samples, which are curated by combining contemporary benchmarks and constitute the largest publicly available dataset till now.
  arXiv  Detail & Related papers  (2024-05-15T14:17:44Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture [23.375515181854254]
  Self-Supervised Learning (SSL) methods can achieve various SAR Automatic Target Recognition (ATR) tasks with pre-training in large-scale unlabeled data. SSL aims to construct supervision signals directly from the data, which minimizes the need for expensive expert annotation. This study investigates an effective SSL method for SAR ATR, which can pave the way for a foundation model in SAR ATR.
  arXiv  Detail & Related papers  (2023-11-26T01:05:55Z)
• Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction [62.955327005837475]
  Image super-resolution (SR) aims to learn a mapping from low-resolution (LR) to high-resolution (HR) using paired HR-LR training images. We present an efficient test-time adaptation framework for SR, named SRTTA, which is able to quickly adapt SR models to test domains with different/unknown degradation types.
  arXiv  Detail & Related papers  (2023-10-29T13:58:57Z)
• Towards Lightweight Super-Resolution with Dual Regression Learning [58.98801753555746]
  Deep neural networks have exhibited remarkable performance in image super-resolution (SR) tasks. The SR problem is typically an ill-posed problem and existing methods would come with several limitations. We propose a dual regression learning scheme to reduce the space of possible SR mappings.
  arXiv  Detail & Related papers  (2022-07-16T12:46:10Z)
• Sparse Signal Models for Data Augmentation in Deep Learning ATR [0.8999056386710496]
  We propose a data augmentation approach to incorporate domain knowledge and improve the generalization power of a data-intensive learning algorithm. We exploit the sparsity of the scattering centers in the spatial domain and the smoothly-varying structure of the scattering coefficients in the azimuthal domain to solve the ill-posed problem of over-parametrized model fitting.
  arXiv  Detail & Related papers  (2020-12-16T21:46:33Z)
• SAR2SAR: a semi-supervised despeckling algorithm for SAR images [3.9490074068698]
  Deep learning algorithm with self-supervision is proposed in this paper: SAR2SAR. The strategy to adapt it to SAR despeckling is presented, based on a compensation of temporal changes and a loss function adapted to the statistics of speckle. Results on real images are discussed, to show the potential of the proposed algorithm.
  arXiv  Detail & Related papers  (2020-06-26T15:07:28Z)
• Unlimited Resolution Image Generation with R2D2-GANs [69.90258455164513]
  We present a novel simulation technique for generating high quality images of any predefined resolution. This method can be used to synthesize sonar scans of size equivalent to those collected during a full-length mission. The data produced is continuous, realistically-looking, and can also be generated at least two times faster than the real speed of acquisition.
  arXiv  Detail & Related papers  (2020-03-02T17:49:32Z)
• Characteristic Regularisation for Super-Resolving Face Images [81.84939112201377]
  Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery. Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data. This renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution. We formulate a method that joins the advantages of conventional SR and UDA models.
  arXiv  Detail & Related papers  (2019-12-30T16:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.