Can SAR improve RSVQA performance?

• URL: http://arxiv.org/abs/2408.15642v1
• Date: Wed, 28 Aug 2024 08:53:20 GMT
• Title: Can SAR improve RSVQA performance?
• Authors: Lucrezia Tosato, Sylvain Lobry, Flora Weissgerber, Laurent Wendling,
• Abstract summary: We study whether Synthetic Aperture Radar (SAR) images can be beneficial to this field. We investigate the classification results of SAR alone and investigate the best method to extract information from SAR data. In the last phase, we investigate how SAR images and a combination of different modalities behave in RSVQA compared to a method only using optical images.
• Score: 1.6249398255272318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Remote sensing visual question answering (RSVQA) has been involved in several research in recent years, leading to an increase in new methods. RSVQA automatically extracts information from satellite images, so far only optical, and a question to automatically search for the answer in the image and provide it in a textual form. In our research, we study whether Synthetic Aperture Radar (SAR) images can be beneficial to this field. We divide our study into three phases which include classification methods and VQA. In the first one, we explore the classification results of SAR alone and investigate the best method to extract information from SAR data. Then, we study the combination of SAR and optical data. In the last phase, we investigate how SAR images and a combination of different modalities behave in RSVQA compared to a method only using optical images. We conclude that adding the SAR modality leads to improved performances, although further research on using SAR data to automatically answer questions is needed as well as more balanced datasets.

Related papers

• Rethinking Image Super-Resolution from Training Data Perspectives [54.28824316574355]
  We investigate the understudied effect of the training data used for image super-resolution (SR) With this, we propose an automated image evaluation pipeline. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance.
  arXiv  Detail & Related papers  (2024-09-01T16:25:04Z)
• SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection [79.23689506129733]
  We establish a new benchmark dataset and an open-source method for large-scale SAR object detection. Our dataset, SARDet-100K, is a result of intense surveying, collecting, and standardizing 10 existing SAR detection datasets. To the best of our knowledge, SARDet-100K is the first COCO-level large-scale multi-class SAR object detection dataset ever created.
  arXiv  Detail & Related papers  (2024-03-11T09:20:40Z)
• Raising the Bar of AI-generated Image Detection with CLIP [50.345365081177555]
  The aim of this work is to explore the potential of pre-trained vision-language models (VLMs) for universal detection of AI-generated images. We develop a lightweight detection strategy based on CLIP features and study its performance in a wide variety of challenging scenarios.
  arXiv  Detail & Related papers  (2023-11-30T21:11:20Z)
• Non-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery [30.590315753622132]
  We explore the "hidden" ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains. We propose a 2-stage low-rank adaptation method, and we call it 2LoRA. In the first stage, the model is adapted using aerial-view regular image data (whose structure matches SAR), followed by the second stage where the base model from the first stage is further adapted using SAR modality data.
  arXiv  Detail & Related papers  (2023-11-29T09:48:01Z)
• CMRxRecon: An open cardiac MRI dataset for the competition of accelerated image reconstruction [62.61209705638161]
  There has been growing interest in deep learning-based CMR imaging algorithms. Deep learning methods require large training datasets. This dataset includes multi-contrast, multi-view, multi-slice and multi-coil CMR imaging data from 300 subjects.
  arXiv  Detail & Related papers  (2023-09-19T15:14:42Z)
• FewSAR: A Few-shot SAR Image Classification Benchmark [17.24173332659616]
  Few-shot learning is one of the significant and hard problems in the field of image classification. FewSAR consists of an open-source Python code library of 15 classic methods in three categories for few-shot SAR image classification. By analyzing the quantitative results and runtime under the same setting, we observe that the accuracy of metric learning methods can achieve the best results.
  arXiv  Detail & Related papers  (2023-06-16T02:35:00Z)
• Confusing Image Quality Assessment: Towards Better Augmented Reality Experience [96.29124666702566]
  We consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. An objective metric termed CFIQA is also proposed to better evaluate the confusing image quality.
  arXiv  Detail & Related papers  (2022-04-11T07:03:06Z)
• Denoising and Optical and SAR Image Classifications Based on Feature Extraction and Sparse Representation [0.0]
  This paper presents a method for denoising, feature extraction and compares classifications of Optical and SAR images. Optical image data have been used by the Remote Sensing workforce to study land use and cover since such data is easily interpretable.
  arXiv  Detail & Related papers  (2021-06-03T14:39:30Z)
• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets [99.92564298432387]
  This work studies Hyperspectral image (HSI) super-resolution (SR) HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples.
  arXiv  Detail & Related papers  (2021-01-19T12:19:53Z)
• Speckle2Void: Deep Self-Supervised SAR Despeckling with Blind-Spot Convolutional Neural Networks [30.410981386006394]
  despeckling is a crucial preliminary step in scene analysis algorithms. Recent success of deep learning envisions a new generation of despeckling techniques. We propose a self-supervised Bayesian despeckling method.
  arXiv  Detail & Related papers  (2020-07-04T11:38:48Z)
• SAR2SAR: a semi-supervised despeckling algorithm for SAR images [3.9490074068698]
  Deep learning algorithm with self-supervision is proposed in this paper: SAR2SAR. The strategy to adapt it to SAR despeckling is presented, based on a compensation of temporal changes and a loss function adapted to the statistics of speckle. Results on real images are discussed, to show the potential of the proposed algorithm.
  arXiv  Detail & Related papers  (2020-06-26T15:07:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.