Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task

• URL: http://arxiv.org/abs/2408.05777v1
• Date: Sun, 11 Aug 2024 14:01:21 GMT
• Title: Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task
• Authors: Hannuo Zhang, Huihui Li, Jiarui Lin, Yujie Zhang, Jianghua Fan, Hang Liu,
• Abstract summary: This letter proposes a GAN-based SAR-to-optical image translation method named Seg-CycleGAN. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of image translation network. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed.
• Score: 12.1644771398574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical remote sensing and Synthetic Aperture Radar(SAR) remote sensing are crucial for earth observation, offering complementary capabilities. While optical sensors provide high-quality images, they are limited by weather and lighting conditions. In contrast, SAR sensors can operate effectively under adverse conditions. This letter proposes a GAN-based SAR-to-optical image translation method named Seg-CycleGAN, designed to enhance the accuracy of ship target translation by leveraging semantic information from a pre-trained semantic segmentation model. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of image translation network, improving the quality of output Optical-styled images. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed. This work suggests broader research and applications for downstream-task-guided frameworks. The code will be available at https://github.com/NPULHH/

Related papers

• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Multi-task SAR Image Processing via GAN-based Unsupervised Manipulation [6.154796320245652]
  Generative Adversarial Networks (GANs) have shown tremendous potential in synthesizing a large number of realistic SAR images. This paper proposes a novel SAR image processing framework called GAN-based Unsupervised Editing (GUE)
  arXiv  Detail & Related papers  (2024-08-02T19:49:30Z)
• Attention Guided Network for Salient Object Detection in Optical Remote Sensing Images [16.933770557853077]
  salient object detection in optical remote sensing images (RSI-SOD) is a very difficult task. We propose a novel Attention Guided Network (AGNet) for SOD in optical RSIs, including position enhancement stage and detail refinement stage. AGNet achieves competitive performance compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-07-05T01:01:03Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• A Dual-fusion Semantic Segmentation Framework With GAN For SAR Images [10.147351262526282]
  A network based on the widely used encoderdecoder architecture is proposed to accomplish the synthetic aperture radar (SAR) images segmentation. With the better representation capability of optical images, we propose to enrich SAR images with generated optical images via the generative adversative network (GAN) trained by numerous SAR and optical images.
  arXiv  Detail & Related papers  (2022-06-02T15:22:29Z)
• Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection [84.52197307286681]
  We propose a novel multitask auto encoding transformation (MAET) model to enhance object detection in a dark environment. In a self-supervision manner, the MAET learns the intrinsic visual structure by encoding and decoding the realistic illumination-degrading transformation. We have achieved the state-of-the-art performance using synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-05-06T16:27:14Z)
• Adjacent Context Coordination Network for Salient Object Detection in Optical Remote Sensing Images [102.75699068451166]
  We propose a novel Adjacent Context Coordination Network (ACCoNet) to explore the coordination of adjacent features in an encoder-decoder architecture for optical RSI-SOD. The proposed ACCoNet outperforms 22 state-of-the-art methods under nine evaluation metrics, and runs up to 81 fps on a single NVIDIA Titan X GPU.
  arXiv  Detail & Related papers  (2022-03-25T14:14:55Z)
• Denoising and Optical and SAR Image Classifications Based on Feature Extraction and Sparse Representation [0.0]
  This paper presents a method for denoising, feature extraction and compares classifications of Optical and SAR images. Optical image data have been used by the Remote Sensing workforce to study land use and cover since such data is easily interpretable.
  arXiv  Detail & Related papers  (2021-06-03T14:39:30Z)
• PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier [50.17500790309477]
  We propose a novel GAN-based multi-task learning (MTL) method for SAR target image generation, called PeaceGAN. PeaceGAN uses both pose angle and target class information, which makes it possible to produce SAR target images of desired target classes at intended pose angles.
  arXiv  Detail & Related papers  (2021-03-29T10:03:09Z)
• Visualization of Deep Transfer Learning In SAR Imagery [0.0]
  We consider transfer learning to leverage deep features from a network trained on an EO ships dataset. By exploring the network activations in the form of class-activation maps, we gain insight on how a deep network interprets a new modality.
  arXiv  Detail & Related papers  (2021-03-20T00:16:15Z)
• Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images [193.77450545067967]
  We propose an end-to-end Dense Attention Fluid Network (DAFNet) for salient object detection in optical remote sensing images (RSIs) A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships. We construct a new and challenging optical RSI dataset for SOD that contains 2,000 images with pixel-wise saliency annotations.
  arXiv  Detail & Related papers  (2020-11-26T06:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.