SAR-TEXT: A Large-Scale SAR Image-Text Dataset Built with SAR-Narrator and Progressive Transfer Learning

• URL: http://arxiv.org/abs/2507.18743v1
• Date: Thu, 24 Jul 2025 18:45:30 GMT
• Title: SAR-TEXT: A Large-Scale SAR Image-Text Dataset Built with SAR-Narrator and Progressive Transfer Learning
• Authors: Xinjun Cheng, Yiguo He, Junjie Zhu, Chunping Qiu, Jun Wang, Qiangjuan Huang, Ke Yang,
• Abstract summary: We construct a large-scale and high-quality SAR image-text dataset consisting of over 130,000 SAR image-text pairs.<n>To verify the effectiveness of the SAR-Text dataset, we conduct experiments on three typical vision-language tasks.<n> SAR-RS-CLIP achieves notable improvements in retrieval performance, boosting average recall by 16.43%.<n>In the captioning task, SAR-RS-CoCa achieves BLEU-4, SPICE, and CIDEr scores exceeding those of the original CoCa model by more than 8x, 4x, and 10x, respectively.
• Score: 15.611051083630862
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision Language Models (VLMs) have achieved remarkable breakthroughs in the field of remote sensing in recent years. Synthetic Aperture Radar (SAR) imagery, with its all-weather capability, is essential in remote sensing, yet the lack of large-scale, high-quality SAR image-text datasets hinders its semantic understanding. In this paper, we construct SAR-Text, a large-scale and high-quality dataset consisting of over 130,000 SAR image-text pairs. To construct the SAR-Text dataset, we design the SAR-Narrator framework, which generates textual descriptions for SAR images through a multi-stage progressive transfer learning strategy. To verify the effectiveness of the SAR-TEXT dataset, we conduct experiments on three typical vision-language tasks: image-text retrieval, image captioning, and visual question answering (VQA). Specifically, we construct three representative models on SAR-TEXT: SAR-RS-CLIP, SAR-RS-CoCa, and SAR-GPT. SAR-RS-CLIP achieves notable improvements in retrieval performance, boosting average recall by 16.43% and 10.54% on the OSdataset-512 and HRSID test sets, respectively. In the captioning task, SAR-RS-CoCa achieves BLEU-4, SPICE, and CIDEr scores exceeding those of the original CoCa model by more than 8x, 4x, and 10x, respectively. In the VQA task, SAR-GPT outperforms baseline and single-stage models on multiple SAR-VQA datasets, demonstrating stronger semantic understanding and reasoning ability, as further confirmed by qualitative results. It is worth noting that, as a flexible captioning tool, SAR-Narrator can be readily adopted by the community to construct larger-scale SAR image-text datasets.

Related papers

• SARLANG-1M: A Benchmark for Vision-Language Modeling in SAR Image Understanding [20.314150537672198]
  Vision-Language Models (VLMs) have demonstrated remarkable success in RGB image understanding, offering powerful open-vocabulary interpretation and flexible language interaction.<n>We introduce SARLANG-1M, a large-scale benchmark tailored for multimodal SAR image understanding, with a primary focus on integrating SAR with textual modality.<n>It features hierarchical resolutions (ranging from 0.1 to 25 meters), fine-grained semantic descriptions (including both concise and detailed captions), diverse remote sensing categories, and multi-task question-answering pairs spanning seven applications and 1,012 question types.
  arXiv  Detail & Related papers  (2025-04-04T08:09:53Z)
• SARChat-Bench-2M: A Multi-Task Vision-Language Benchmark for SAR Image Interpretation [12.32553804641971]
  Vision language models (VLMs) have made remarkable progress in natural language processing and image understanding.<n>This paper innovatively proposes the first large-scale multimodal dialogue dataset for SAR images, named SARChat-2M.
  arXiv  Detail & Related papers  (2025-02-12T07:19:36Z)
• SAR Strikes Back: A New Hope for RSVQA [1.6249398255272318]
  We present a dataset that allows for the introduction of SAR images in the RSVQA framework.<n>SAR images capture electromagnetic information from the scene, and are less affected by atmospheric conditions, such as clouds.<n>We show that SAR data offers additional information when fused with the optical modality, particularly for questions related to specific land cover classes, such as water areas.
  arXiv  Detail & Related papers  (2025-01-14T14:07:48Z)
• Rethinking Image Super-Resolution from Training Data Perspectives [54.28824316574355]
  We investigate the understudied effect of the training data used for image super-resolution (SR) With this, we propose an automated image evaluation pipeline. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance.
  arXiv  Detail & Related papers  (2024-09-01T16:25:04Z)
• Can SAR improve RSVQA performance? [1.6249398255272318]
  We study whether Synthetic Aperture Radar (SAR) images can be beneficial to this field. We investigate the classification results of SAR alone and investigate the best method to extract information from SAR data. In the last phase, we investigate how SAR images and a combination of different modalities behave in RSVQA compared to a method only using optical images.
  arXiv  Detail & Related papers  (2024-08-28T08:53:20Z)
• Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval [50.72924579220149]
  Composed Image Retrieval (CIR) is a task that retrieves images similar to a query, based on a provided textual modification. Current techniques rely on supervised learning for CIR models using labeled triplets of the reference image, text, target image. We propose a new semi-supervised CIR approach where we search for a reference and its related target images in auxiliary data.
  arXiv  Detail & Related papers  (2024-04-23T21:00:22Z)
• SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection [79.23689506129733]
  We establish a new benchmark dataset and an open-source method for large-scale SAR object detection. Our dataset, SARDet-100K, is a result of intense surveying, collecting, and standardizing 10 existing SAR detection datasets. To the best of our knowledge, SARDet-100K is the first COCO-level large-scale multi-class SAR object detection dataset ever created.
  arXiv  Detail & Related papers  (2024-03-11T09:20:40Z)
• Retrieval-based Disentangled Representation Learning with Natural Language Supervision [61.75109410513864]
  We present Vocabulary Disentangled Retrieval (VDR), a retrieval-based framework that harnesses natural language as proxies of the underlying data variation to drive disentangled representation learning. Our approach employ a bi-encoder model to represent both data and natural language in a vocabulary space, enabling the model to distinguish intrinsic dimensions that capture characteristics within data through its natural language counterpart, thus disentanglement.
  arXiv  Detail & Related papers  (2022-12-15T10:20:42Z)
• The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion [14.45289690639374]
  We publish the QXS-SAROPT dataset to foster deep learning research in SAR-optical data fusion. We show exemplary results for two representative applications, namely SAR-optical image matching and SAR ship detection boosted by cross-modal information from optical images.
  arXiv  Detail & Related papers  (2021-03-15T10:22:46Z)
• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets [99.92564298432387]
  This work studies Hyperspectral image (HSI) super-resolution (SR) HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples.
  arXiv  Detail & Related papers  (2021-01-19T12:19:53Z)
• TAP: Text-Aware Pre-training for Text-VQA and Text-Caption [75.44716665758415]
  We propose Text-Aware Pre-training (TAP) for Text-VQA and Text-Caption tasks. TAP explicitly incorporates scene text (generated from OCR engines) in pre-training. Our approach outperforms the state of the art by large margins on multiple tasks.
  arXiv  Detail & Related papers  (2020-12-08T18:55:21Z)
• On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances and Million-AID [57.71601467271486]
  This article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. We first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million-AID, a new large-scale benchmark dataset.
  arXiv  Detail & Related papers  (2020-06-22T17:59:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.