Cross-modal learning for SAR target recognition using optical vision foundation models

• URL: http://arxiv.org/abs/2609.07753v1
• Date: Mon, 07 Sep 2026 16:54:30 GMT
• Title: Cross-modal learning for SAR target recognition using optical vision foundation models
• Abstract summary: We investigate how vision foundation models trained on optical data can provide class level supervision for SAR classification.<n>A SAR model is then trained to classify SAR images while aligning its embeddings to the corresponding EO class prototype.<n>We evaluate our approach on the UNICORNv2 dataset, an EO and SAR dataset of civilian vehicles with heavily speckled images.
• Score: 10.266600680575166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthetic Aperture Radar (SAR) is an important modality in a wide range of imaging applications due to its versatile, long range and near all weather operating capabilities. However, Automatic Target Recognition (ATR) remains a challenging problem due to limited labelled data, the strong speckle in SAR images and the significant domain gap between SAR and more abundant optical imagery. In contrast, electro-optical (EO) imagery benefits from massive datasets, clearer visual structure and powerful foundation models. In this work, we investigate how vision foundation models trained on optical data can provide class level supervision for SAR classification. We propose a cross-modal EO to SAR prototype alignment framework in which a frozen EO encoder, based on a DINOv3 vision foundation model, is used to construct class level optical prototypes without requiring strict EO/SAR pairs. A SAR model is then trained to classify SAR images while aligning its embeddings to the corresponding EO class prototype. At inference time, the SAR model operates independently, without access to optical imagery. We evaluate our approach on the UNICORNv2 dataset, an EO and SAR dataset of civilian vehicles with heavily speckled images and severe class imbalance. EO prototype alignment improves SAR classification accuracy over frozen DINOv3, SAR only finetuning and unpaired distribution alignment baselines, and t-SNE visualizations provide qualitative evidence of clearer separation among classes in the trained SAR embedding space. These results suggest that optical vision foundation models, despite being trained on visible spectrum imagery, provide transferable information for SAR image classification, offering a practical method for using large scale pretrained vision foundation models across challenging sensing modalities.

Related papers

• SARCLIP: A Vision Language Foundation Model for Semantic Understanding and Target Recognition in SAR Imagery [46.87845911116779]
  We introduce SARCLIP, the first vision language foundation model tailored for the SAR domain.<n>SARCLIP is trained using a contrastive vision language learning approach by domain transferring strategy.<n>Experiments on image-text retrieval and zero-shot classification tasks demonstrate the superior performance of SARCLIP.
  arXiv  Detail & Related papers  (2025-10-26T13:04:50Z)
• Cross-View Open-Vocabulary Object Detection in Aerial Imagery [48.851422992413184]
  We propose a novel framework for adapting open-vocabulary representations from ground-view images to solve object detection in aerial imagery.<n>The method introduces contrastive image-to-image alignment to enhance the similarity between aerial and ground-view embeddings.<n>Our open-vocabulary model achieves improvements of +6.32 mAP on DOTAv2, +4.16 mAP on VisDrone (Images), and +3.46 mAP on HRRSD in the zero-shot setting.
  arXiv  Detail & Related papers  (2025-10-04T16:12:03Z)
• Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images [51.74614065919118]
  This paper introduces SegEarth-OV, the first framework for annotation-free open-vocabulary segmentation of RS images.<n>We propose SimFeatUp, a universal upsampler that robustly restores high-resolution spatial details from coarse features.<n>We also present a simple yet effective Global Bias Alleviation operation to subtract the inherent global context from patch features.
  arXiv  Detail & Related papers  (2025-08-25T14:22:57Z)
• Efficient Self-Supervised Learning for Earth Observation via Dynamic Dataset Curation [67.23953699167274]
  Self-supervised learning (SSL) has enabled the development of vision foundation models for Earth Observation (EO)<n>In EO, this challenge is amplified by the redundancy and heavy-tailed distributions common in satellite imagery.<n>We propose a dynamic dataset pruning strategy designed to improve SSL pre-training by maximizing dataset diversity and balance.
  arXiv  Detail & Related papers  (2025-04-09T15:13:26Z)
• Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models [4.6570959687411975]
  Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks.<n>This study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets.
  arXiv  Detail & Related papers  (2025-03-25T16:05:49Z)
• SAR-W-MixMAE: SAR Foundation Model Training Using Backscatter Power Weighting [3.618534280726541]
  Foundation model approaches such as masked auto-encoders (MAE) or its variations are now being successfully applied to satellite imagery.<n>Due to difficulty in semantic labeling to create datasets and higher noise content with respect to optical images, Synthetic Aperture Radar (SAR) data has not been explored a lot in the field for foundation models.<n>In this work, we explored masked auto-encoder, specifically MixMAE on Sentinel-1 SAR images and its impact on SAR image classification tasks.
  arXiv  Detail & Related papers  (2025-03-03T05:09:44Z)
• Diffusion Model Based Visual Compensation Guidance and Visual Difference Analysis for No-Reference Image Quality Assessment [78.21609845377644]
  We propose a novel class of state-of-the-art (SOTA) generative model, which exhibits the capability to model intricate relationships.<n>We devise a new diffusion restoration network that leverages the produced enhanced image and noise-containing images.<n>Two visual evaluation branches are designed to comprehensively analyze the obtained high-level feature information.
  arXiv  Detail & Related papers  (2024-02-22T09:39:46Z)
• Non-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery [30.590315753622132]
  We explore the "hidden" ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains. We propose a 2-stage low-rank adaptation method, and we call it 2LoRA. In the first stage, the model is adapted using aerial-view regular image data (whose structure matches SAR), followed by the second stage where the base model from the first stage is further adapted using SAR modality data.
  arXiv  Detail & Related papers  (2023-11-29T09:48:01Z)
• PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier [50.17500790309477]
  We propose a novel GAN-based multi-task learning (MTL) method for SAR target image generation, called PeaceGAN. PeaceGAN uses both pose angle and target class information, which makes it possible to produce SAR target images of desired target classes at intended pose angles.
  arXiv  Detail & Related papers  (2021-03-29T10:03:09Z)
• Visualization of Deep Transfer Learning In SAR Imagery [0.0]
  We consider transfer learning to leverage deep features from a network trained on an EO ships dataset. By exploring the network activations in the form of class-activation maps, we gain insight on how a deep network interprets a new modality.
  arXiv  Detail & Related papers  (2021-03-20T00:16:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.