1st Place Solution to MultiEarth 2023 Challenge on Multimodal SAR-to-EO Image Translation

• URL: http://arxiv.org/abs/2306.12626v1
• Date: Thu, 22 Jun 2023 01:32:30 GMT
• Title: 1st Place Solution to MultiEarth 2023 Challenge on Multimodal SAR-to-EO Image Translation
• Authors: Jingi Ju, Hyeoncheol Noh, Minwoo Kim, Dong-Geol Choi
• Abstract summary: Multimodal Learning for Earth and Environment Workshop (MultiEarth 2023) aims to harness the substantial amount of remote sensing data gathered over extensive periods for the monitoring and analysis of Earth's ecosystems'health. The subtask, Multimodal SAR-to-EO Image Translation, involves the use of robust SAR data, even under adverse weather and lighting conditions, transforming it into high-quality, clear, and visually appealing EO data. In the final evaluation, the team 'CDRL' achieved an MAE of 0.07313, securing the top rank on the leaderboard.
• Score: 3.8424737607413153
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Multimodal Learning for Earth and Environment Workshop (MultiEarth 2023) aims to harness the substantial amount of remote sensing data gathered over extensive periods for the monitoring and analysis of Earth's ecosystems'health. The subtask, Multimodal SAR-to-EO Image Translation, involves the use of robust SAR data, even under adverse weather and lighting conditions, transforming it into high-quality, clear, and visually appealing EO data. In the context of the SAR2EO task, the presence of clouds or obstructions in EO data can potentially pose a challenge. To address this issue, we propose the Clean Collector Algorithm (CCA), designed to take full advantage of this cloudless SAR data and eliminate factors that may hinder the data learning process. Subsequently, we applied pix2pixHD for the SAR-to-EO translation and Restormer for image enhancement. In the final evaluation, the team 'CDRL' achieved an MAE of 0.07313, securing the top rank on the leaderboard.

Related papers

• C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation [23.63992950769041]
  C-DiffSET is a framework leveraging pretrained Latent Diffusion Model (LDM) extensively trained on natural images. Remarkably, we find that the pretrained VAE encoder aligns SAR and EO images in the same latent space, even with varying noise levels in SAR inputs.
  arXiv  Detail & Related papers  (2024-11-16T12:28:40Z)
• AllClear: A Comprehensive Dataset and Benchmark for Cloud Removal in Satellite Imagery [38.538566901989526]
  We introduce the largest public dataset -- $textitAllClear$ for cloud removal, featuring 23,742 globally distributed regions of interest. This dataset aims to provide comprehensive coverage of the Earth's surface and promote better cloud removal results.
  arXiv  Detail & Related papers  (2024-10-31T12:52:52Z)
• Rethinking Image Super-Resolution from Training Data Perspectives [54.28824316574355]
  We investigate the understudied effect of the training data used for image super-resolution (SR) With this, we propose an automated image evaluation pipeline. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance.
  arXiv  Detail & Related papers  (2024-09-01T16:25:04Z)
• SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection [79.23689506129733]
  We establish a new benchmark dataset and an open-source method for large-scale SAR object detection. Our dataset, SARDet-100K, is a result of intense surveying, collecting, and standardizing 10 existing SAR detection datasets. To the best of our knowledge, SARDet-100K is the first COCO-level large-scale multi-class SAR object detection dataset ever created.
  arXiv  Detail & Related papers  (2024-03-11T09:20:40Z)
• Improved Flood Insights: Diffusion-Based SAR to EO Image Translation [4.994315051443544]
  This paper introduces a novel framework, Diffusion-Based SAR to EO Image Translation (DSE) The DSE framework converts SAR images into EO images, thereby enhancing the interpretability of flood insights for humans. Experimental results on the Sen1Floods11 and SEN12-FLOOD datasets confirm that the DSE framework not only delivers enhanced visual information but also improves performance.
  arXiv  Detail & Related papers  (2023-07-14T02:19:23Z)
• SAR2EO: A High-resolution Image Translation Framework with Denoising Enhancement [10.11898520476921]
  We propose a framework, SAR2EO, to complete the conversion from low-resolution images to high-resolution images. Firstly, to generate high-quality EO images, we adopt the coarse-to-fine generator, multi-scale discriminators, and improved adversarial loss in the pix2pixHD model. Secondly, we introduce a denoising module to remove the noise in SAR images, which helps to suppress the noise while preserving the structural information of the images.
  arXiv  Detail & Related papers  (2023-04-08T03:39:51Z)
• A Multi-purpose Real Haze Benchmark with Quantifiable Haze Levels and Ground Truth [61.90504318229845]
  This paper introduces the first paired real image benchmark dataset with hazy and haze-free images, and in-situ haze density measurements. This dataset was produced in a controlled environment with professional smoke generating machines that covered the entire scene. A subset of this dataset has been used for the Object Detection in Haze Track of CVPR UG2 2022 challenge.
  arXiv  Detail & Related papers  (2022-06-13T19:14:06Z)
• Real-World Image Super-Resolution by Exclusionary Dual-Learning [98.36096041099906]
  Real-world image super-resolution is a practical image restoration problem that aims to obtain high-quality images from in-the-wild input. Deep learning-based methods have achieved promising restoration quality on real-world image super-resolution datasets. We propose Real-World image Super-Resolution by Exclusionary Dual-Learning (RWSR-EDL) to address the feature diversity in perceptual- and L1-based cooperative learning.
  arXiv  Detail & Related papers  (2022-06-06T13:28:15Z)
• The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion [14.45289690639374]
  We publish the QXS-SAROPT dataset to foster deep learning research in SAR-optical data fusion. We show exemplary results for two representative applications, namely SAR-optical image matching and SAR ship detection boosted by cross-modal information from optical images.
  arXiv  Detail & Related papers  (2021-03-15T10:22:46Z)
• Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges [124.48654341780431]
  We present a large-scale dataset of Object deTection in Aerial images (DOTA) and comprehensive baselines for ODAI. The proposed DOTA dataset contains 1,793,658 object instances of 18 categories of oriented-bounding-box annotations collected from 11,268 aerial images. We build baselines covering 10 state-of-the-art algorithms with over 70 configurations, where the speed and accuracy performances of each model have been evaluated.
  arXiv  Detail & Related papers  (2021-02-24T11:20:55Z)
• HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery [55.253395881190436]
  Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem. This is important for satellite monitoring of human impact on the planet. We present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion.
  arXiv  Detail & Related papers  (2020-02-15T22:17:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.