SARFormer -- An Acquisition Parameter Aware Vision Transformer for Synthetic Aperture Radar Data

• URL: http://arxiv.org/abs/2504.08441v1
• Date: Fri, 11 Apr 2025 11:06:12 GMT
• Title: SARFormer -- An Acquisition Parameter Aware Vision Transformer for Synthetic Aperture Radar Data
• Authors: Jonathan Prexl, Michael Recla, Michael Schmitt,
• Abstract summary: We introduce a modified Vision Transformer (ViT) architecture designed for processing one or multiple synthetic aperture radar (SAR) images.<n>We propose an acquisition parameter encoding module that significantly guides the learning process.<n>Our approach achieves up to 17% improvement in terms of RMSE over baseline models.
• Score: 1.2926587870771544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This manuscript introduces SARFormer, a modified Vision Transformer (ViT) architecture designed for processing one or multiple synthetic aperture radar (SAR) images. Given the complex image geometry of SAR data, we propose an acquisition parameter encoding module that significantly guides the learning process, especially in the case of multiple images, leading to improved performance on downstream tasks. We further explore self-supervised pre-training, conduct experiments with limited labeled data, and benchmark our contribution and adaptations thoroughly in ablation experiments against a baseline, where the model is tested on tasks such as height reconstruction and segmentation. Our approach achieves up to 17% improvement in terms of RMSE over baseline models

Related papers

• Quantitative Comparison of Fine-Tuning Techniques for Pretrained Latent Diffusion Models in the Generation of Unseen SAR Image Concepts [0.0]
  This work investigates the adaptation of large pre-trained latent diffusion models to a radically new imaging domain: Synthetic Aperture Radar (SAR)<n>We explore and compare multiple fine-tuning strategies, including full model fine-tuning and parameter-efficient approaches like Low-Rank Adaptation (LoRA)<n>Our results show that a hybrid tuning strategy yields the best performance, while LoRA-based partial tuning of the text encoder, combined with embedding learning of the SAR> token, suffices to preserve prompt alignment.
  arXiv  Detail & Related papers  (2025-06-16T09:48:01Z)
• SAR-W-MixMAE: SAR Foundation Model Training Using Backscatter Power Weighting [3.618534280726541]
  Foundation model approaches such as masked auto-encoders (MAE) or its variations are now being successfully applied to satellite imagery.<n>Due to difficulty in semantic labeling to create datasets and higher noise content with respect to optical images, Synthetic Aperture Radar (SAR) data has not been explored a lot in the field for foundation models.<n>In this work, we explored masked auto-encoder, specifically MixMAE on Sentinel-1 SAR images and its impact on SAR image classification tasks.
  arXiv  Detail & Related papers  (2025-03-03T05:09:44Z)
• Rethinking Image Super-Resolution from Training Data Perspectives [54.28824316574355]
  We investigate the understudied effect of the training data used for image super-resolution (SR) With this, we propose an automated image evaluation pipeline. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance.
  arXiv  Detail & Related papers  (2024-09-01T16:25:04Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• ClassWise-SAM-Adapter: Parameter Efficient Fine-tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation [6.229326337093342]
  Segment Anything Model (SAM) excels in various segmentation scenarios relying on semantic information and generalization ability. The ClassWiseSAM-Adapter (CWSAM) is designed to adapt the high-performing SAM for landcover classification on space-borne Synthetic Aperture Radar (SAR) images. CWSAM showcases enhanced performance with fewer computing resources.
  arXiv  Detail & Related papers  (2024-01-04T15:54:45Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• Zero-shot Composed Text-Image Retrieval [72.43790281036584]
  We consider the problem of composed image retrieval (CIR) It aims to train a model that can fuse multi-modal information, e.g., text and images, to accurately retrieve images that match the query, extending the user's expression ability.
  arXiv  Detail & Related papers  (2023-06-12T17:56:01Z)
• Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model [97.9548609175831]
  We resort to plain vision transformers with about 100 million parameters and make the first attempt to propose large vision models customized for remote sensing tasks. Specifically, to handle the large image size and objects of various orientations in RS images, we propose a new rotated varied-size window attention. Experiments on detection tasks demonstrate the superiority of our model over all state-of-the-art models, achieving 81.16% mAP on the DOTA-V1.0 dataset.
  arXiv  Detail & Related papers  (2022-08-08T09:08:40Z)
• An Empirical Study of Remote Sensing Pretraining [117.90699699469639]
  We conduct an empirical study of remote sensing pretraining (RSP) on aerial images. RSP can help deliver distinctive performances in scene recognition tasks. RSP mitigates the data discrepancies of traditional ImageNet pretraining on RS images, but it may still suffer from task discrepancies.
  arXiv  Detail & Related papers  (2022-04-06T13:38:11Z)
• Patch Similarity Aware Data-Free Quantization for Vision Transformers [2.954890575035673]
  We propose PSAQ-ViT, a Patch Similarity Aware data-free Quantization framework for Vision Transformers. We analyze the self-attention module's properties and reveal a general difference (patch similarity) in its processing of Gaussian noise and real images. Experiments and ablation studies are conducted on various benchmarks to validate the effectiveness of PSAQ-ViT.
  arXiv  Detail & Related papers  (2022-03-04T11:47:20Z)
• Plug-In Inversion: Model-Agnostic Inversion for Vision with Data Augmentations [61.95114821573875]
  We introduce Plug-In Inversion, which relies on a simple set of augmentations and does not require excessive hyper- parameter tuning. We illustrate the practicality of our approach by inverting Vision Transformers (ViTs) and Multi-Layer Perceptrons (MLPs) trained on the ImageNet dataset.
  arXiv  Detail & Related papers  (2022-01-31T02:12:45Z)
• Deep-Learning-Based Single-Image Height Reconstruction from Very-High-Resolution SAR Intensity Data [1.7894377200944511]
  We present the first-ever demonstration of deep learning-based single image height prediction for the other important sensor modality in remote sensing: synthetic aperture radar (SAR) data. Besides the adaptation of a convolutional neural network (CNN) architecture for SAR intensity images, we present a workflow for the generation of training data. Since we put a particular emphasis on transferability, we are able to confirm that deep learning-based single-image height estimation is not only possible, but also transfers quite well to unseen data.
  arXiv  Detail & Related papers  (2021-11-03T08:20:03Z)
• Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentation [0.6445605125467573]
  We propose multi-modal representation learning for SAR semantic segmentation. Unlike previous studies, our method jointly uses EO imagery, SAR imagery, and a label mask. Several experiments show that our approach is superior to the existing methods in model performance, sample efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2021-08-31T23:55:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.