Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models

• URL: http://arxiv.org/abs/2503.19798v1
• Date: Tue, 25 Mar 2025 16:05:49 GMT
• Title: Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models
• Authors: Ruixi You, Hecheng Jia, Feng Xu,
• Abstract summary: Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks.<n>This study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets.
• Score: 4.6570959687411975
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthetic Aperture Radar (SAR) imagery provides all-weather, all-day, and high-resolution imaging capabilities but its unique imaging mechanism makes interpretation heavily reliant on expert knowledge, limiting interpretability, especially in complex target tasks. Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks. Most existing research focuses on scene-level translation, with limited work on object-level translation due to the scarcity of paired data and the challenge of accurately preserving contour and texture details. To address these issues, this study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets. This framework introduces supervision on target class and azimuth angle via keypoints, along with a training strategy for unpaired data. Based on the classifier-free guidance diffusion architecture, a class-angle guidance module (CAGM) is designed to integrate class and angle information into the diffusion generation process. Furthermore, adversarial loss and consistency loss are employed to improve image fidelity and detail quality, tailored for aircraft targets. During sampling, aided by a pre-trained keypoint detector, the model eliminates the requirement for manually labeled class and azimuth information, enabling automated SAR-to-optical translation. Experimental results demonstrate that the proposed method outperforms existing approaches across multiple metrics, providing an efficient and effective solution for object-level SAR-to-optical translation and downstream tasks. Moreover, the method exhibits strong zero-shot generalization to untrained aircraft types with the assistance of the keypoint detector.

Related papers

• Bottom-Up Scattering Information Perception Network for SAR target recognition [9.694730272245849]
  This paper proposes a novel bottom-up scattering information perception network for more interpretable target recognition. First, the localized scattering perceptron is proposed to replace the backbone feature extractor based on CNN networks. Second, an unsupervised scattering part feature extraction model is proposed to robustly characterize the target scattering part information. Third, by aggregating the knowledge of target parts to form the complete target description, the interpretability and discriminative ability of the model is improved.
  arXiv  Detail & Related papers  (2025-04-07T07:15:08Z)
• Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task [12.1644771398574]
  This letter proposes a GAN-based SAR-to-optical image translation method named Seg-CycleGAN. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of image translation network. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed.
  arXiv  Detail & Related papers  (2024-08-11T14:01:21Z)
• SAFE: a SAR Feature Extractor based on self-supervised learning and masked Siamese ViTs [5.961207817077044]
  We propose a novel self-supervised learning framework based on masked Siamese Vision Transformers to create a General SAR Feature Extractor coined SAFE. Our method leverages contrastive learning principles to train a model on unlabeled SAR data, extracting robust and generalizable features. We introduce tailored data augmentation techniques specific to SAR imagery, such as sub-aperture decomposition and despeckling. Our network competes with or surpasses other state-of-the-art methods in few-shot classification and segmentation tasks, even without being trained on the sensors used for the evaluation.
  arXiv  Detail & Related papers  (2024-06-30T23:11:20Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for Advanced Object Detection [55.2480439325792]
  We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head. The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement. This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
  arXiv  Detail & Related papers  (2023-11-21T19:49:13Z)
• You Only Train Once: A Unified Framework for Both Full-Reference and No-Reference Image Quality Assessment [45.62136459502005]
  We propose a network to perform full reference (FR) and no reference (NR) IQA. We first employ an encoder to extract multi-level features from input images. A Hierarchical Attention (HA) module is proposed as a universal adapter for both FR and NR inputs. A Semantic Distortion Aware (SDA) module is proposed to examine feature correlations between shallow and deep layers of the encoder.
  arXiv  Detail & Related papers  (2023-10-14T11:03:04Z)
• Exploring Resolution and Degradation Clues as Self-supervised Signal for Low Quality Object Detection [77.3530907443279]
  We propose a novel self-supervised framework to detect objects in degraded low resolution images. Our methods has achieved superior performance compared with existing methods when facing variant degradation situations.
  arXiv  Detail & Related papers  (2022-08-05T09:36:13Z)
• Knowledge Distillation for Oriented Object Detection on Aerial Images [1.827510863075184]
  We present a model compression method for rotated object detection on aerial images by knowledge distillation, namely KD-RNet. The experimental result on a large-scale aerial object detection dataset (DOTA) demonstrates that the proposed KD-RNet model can achieve improved mean-average precision (mAP) with reduced number of parameters, at the same time, KD-RNet boost the performance on providing high quality detections with higher overlap with groundtruth annotations.
  arXiv  Detail & Related papers  (2022-06-20T14:24:16Z)
• Learning Efficient Representations for Enhanced Object Detection on Large-scene SAR Images [16.602738933183865]
  It is a challenging problem to detect and recognize targets on complex large-scene Synthetic Aperture Radar (SAR) images. Recently developed deep learning algorithms can automatically learn the intrinsic features of SAR images. We propose an efficient and robust deep learning based target detection method.
  arXiv  Detail & Related papers  (2022-01-22T03:25:24Z)
• RRNet: Relational Reasoning Network with Parallel Multi-scale Attention for Salient Object Detection in Optical Remote Sensing Images [82.1679766706423]
  Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. We propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs. Our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2021-10-27T07:18:32Z)
• PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier [50.17500790309477]
  We propose a novel GAN-based multi-task learning (MTL) method for SAR target image generation, called PeaceGAN. PeaceGAN uses both pose angle and target class information, which makes it possible to produce SAR target images of desired target classes at intended pose angles.
  arXiv  Detail & Related papers  (2021-03-29T10:03:09Z)
• Sparse Signal Models for Data Augmentation in Deep Learning ATR [0.8999056386710496]
  We propose a data augmentation approach to incorporate domain knowledge and improve the generalization power of a data-intensive learning algorithm. We exploit the sparsity of the scattering centers in the spatial domain and the smoothly-varying structure of the scattering coefficients in the azimuthal domain to solve the ill-posed problem of over-parametrized model fitting.
  arXiv  Detail & Related papers  (2020-12-16T21:46:33Z)
• Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images [193.77450545067967]
  We propose an end-to-end Dense Attention Fluid Network (DAFNet) for salient object detection in optical remote sensing images (RSIs) A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships. We construct a new and challenging optical RSI dataset for SOD that contains 2,000 images with pixel-wise saliency annotations.
  arXiv  Detail & Related papers  (2020-11-26T06:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.