Comparison of convolutional neural networks for cloudy optical images reconstruction from single or multitemporal joint SAR and optical images

• URL: http://arxiv.org/abs/2204.00424v1
• Date: Fri, 1 Apr 2022 13:31:23 GMT
• Title: Comparison of convolutional neural networks for cloudy optical images reconstruction from single or multitemporal joint SAR and optical images
• Authors: R\'emi Cresson, Nicolas Nar\c{c}on, Raffaele Gaetano, Aurore Dupuis, Yannick Tanguy, St\'ephane May, and Benjamin Commandre
• Abstract summary: We focus on the evaluation of convolutional neural networks that use jointly SAR and optical images to retrieve the missing contents in one single polluted optical image. We propose a simple framework that eases the creation of datasets for the training of deep nets targeting optical image reconstruction. We show how space partitioning data structures help to query samples in terms of cloud coverage, relative acquisition date, pixel validity and relative proximity between SAR and optical images.
• Score: 0.21079694661943604
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing availability of optical and synthetic aperture radar (SAR) images thanks to the Sentinel constellation, and the explosion of deep learning, new methods have emerged in recent years to tackle the reconstruction of optical images that are impacted by clouds. In this paper, we focus on the evaluation of convolutional neural networks that use jointly SAR and optical images to retrieve the missing contents in one single polluted optical image. We propose a simple framework that ease the creation of datasets for the training of deep nets targeting optical image reconstruction, and for the validation of machine learning based or deterministic approaches. These methods are quite different in terms of input images constraints, and comparing them is a problematic task not addressed in the literature. We show how space partitioning data structures help to query samples in terms of cloud coverage, relative acquisition date, pixel validity and relative proximity between SAR and optical images. We generate several datasets to compare the reconstructed images from networks that use a single pair of SAR and optical image, versus networks that use multiple pairs, and a traditional deterministic approach performing interpolation in temporal domain.

Related papers

• OADAT: Experimental and Synthetic Clinical Optoacoustic Data for Standardized Image Processing [62.993663757843464]
  Optoacoustic (OA) imaging is based on excitation of biological tissues with nanosecond-duration laser pulses followed by detection of ultrasound waves generated via light-absorption-mediated thermoelastic expansion. OA imaging features a powerful combination between rich optical contrast and high resolution in deep tissues. No standardized datasets generated with different types of experimental set-up and associated processing methods are available to facilitate advances in broader applications of OA in clinical settings.
  arXiv  Detail & Related papers  (2022-06-17T08:11:26Z)
• Transformer-based SAR Image Despeckling [53.99620005035804]
  We introduce a transformer-based network for SAR image despeckling. The proposed despeckling network comprises of a transformer-based encoder which allows the network to learn global dependencies between different image regions. Experiments show that the proposed method achieves significant improvements over traditional and convolutional neural network-based despeckling methods.
  arXiv  Detail & Related papers  (2022-01-23T20:09:01Z)
• Enhancing Photorealism Enhancement [83.88433283714461]
  We present an approach to enhancing the realism of synthetic images using a convolutional network. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We report substantial gains in stability and realism in comparison to recent image-to-image translation methods.
  arXiv  Detail & Related papers  (2021-05-10T19:00:49Z)
• Learning optical flow from still images [53.295332513139925]
  We introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. We virtually move the camera in the reconstructed environment with known motion vectors and rotation angles. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data.
  arXiv  Detail & Related papers  (2021-04-08T17:59:58Z)
• AS-Net: Fast Photoacoustic Reconstruction with Multi-feature Fusion from Sparse Data [1.7237160821929758]
  Photoacoustic imaging is capable of acquiring high contrast images of optical absorption at depths much greater than traditional optical imaging techniques. In this paper, we employ a novel signal processing method to make sparse PA raw data more suitable for the neural network. We then propose Attention Steered Network (AS-Net) for PA reconstruction with multi-feature fusion.
  arXiv  Detail & Related papers  (2021-01-22T03:49:30Z)
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search [65.79109790446257]
  Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior. We propose to search for neural architectures that capture stronger image priors. We search for an improved network by leveraging an existing neural architecture search algorithm.
  arXiv  Detail & Related papers  (2020-08-26T17:59:36Z)
• Generative Adversarial Networks for Synthesizing InSAR Patches [15.260123615399035]
  Generative Adversarial Networks (GANs) have been employed with certain success for image translation tasks between optical and real-valued SAR intensity imagery. The synthesis of artificial complex-valued InSAR image stacks asks for, besides good perceptual quality, more stringent quality metrics like phase noise and phase coherence. This paper provides a signal processing model of generative CNN structures, describes effects influencing those quality metrics and presents a mapping scheme of complex-valued data to given CNN structures.
  arXiv  Detail & Related papers  (2020-08-03T20:51:01Z)
• Speckle2Void: Deep Self-Supervised SAR Despeckling with Blind-Spot Convolutional Neural Networks [30.410981386006394]
  despeckling is a crucial preliminary step in scene analysis algorithms. Recent success of deep learning envisions a new generation of despeckling techniques. We propose a self-supervised Bayesian despeckling method.
  arXiv  Detail & Related papers  (2020-07-04T11:38:48Z)
• A Convolutional Neural Network with Parallel Multi-Scale Spatial Pooling to Detect Temporal Changes in SAR Images [43.56177583903999]
  In synthetic aperture radar (SAR) image change detection, it is quite challenging to exploit the changing information from the noisy difference image. We propose a multi-scale spatial pooling (MSSP) network to exploit the changed information from the noisy difference image.
  arXiv  Detail & Related papers  (2020-05-22T03:37:30Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)
• Towards Deep Unsupervised SAR Despeckling with Blind-Spot Convolutional Neural Networks [30.410981386006394]
  Deep learning techniques have outperformed classical model-based despeckling algorithms. In this paper, we propose a self-supervised Bayesian despeckling method. We show that the performance of the proposed network is very close to the supervised training approach on synthetic data and competitive on real data.
  arXiv  Detail & Related papers  (2020-01-15T12:21:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.