SAR-to-Optical Image Translation via Thermodynamics-inspired Network

• URL: http://arxiv.org/abs/2305.13839v1
• Date: Tue, 23 May 2023 09:02:33 GMT
• Title: SAR-to-Optical Image Translation via Thermodynamics-inspired Network
• Authors: Mingjin Zhang, Jiamin Xu, Chengyu He, Wenteng Shang, Yunsong Li, and Xinbo Gao
• Abstract summary: A Thermodynamics-inspired Network for SAR-to-Optical Image Translation (S2O-TDN) is proposed in this paper. S2O-TDN follows an explicit design principle derived from thermodynamic theory and enjoys the advantage of explainability. Experiments on the public SEN1-2 dataset show the advantages of the proposed S2O-TDN over the current methods with more delicate textures and higher quantitative results.
• Score: 68.71771171637677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthetic aperture radar (SAR) is prevalent in the remote sensing field but is difficult to interpret in human visual perception. Recently, SAR-to-optical (S2O) image conversion methods have provided a prospective solution for interpretation. However, since there is a huge domain difference between optical and SAR images, they suffer from low image quality and geometric distortion in the produced optical images. Motivated by the analogy between pixels during the S2O image translation and molecules in a heat field, Thermodynamics-inspired Network for SAR-to-Optical Image Translation (S2O-TDN) is proposed in this paper. Specifically, we design a Third-order Finite Difference (TFD) residual structure in light of the TFD equation of thermodynamics, which allows us to efficiently extract inter-domain invariant features and facilitate the learning of the nonlinear translation mapping. In addition, we exploit the first law of thermodynamics (FLT) to devise an FLT-guided branch that promotes the state transition of the feature values from the unstable diffusion state to the stable one, aiming to regularize the feature diffusion and preserve image structures during S2O image translation. S2O-TDN follows an explicit design principle derived from thermodynamic theory and enjoys the advantage of explainability. Experiments on the public SEN1-2 dataset show the advantages of the proposed S2O-TDN over the current methods with more delicate textures and higher quantitative results.

Related papers

• Zak Phase Induced Topological Nonreciprocity [21.20214270947359]
  We study the optical nonreciprocity of the 1D Su-Schrieffer-Heeger superradiance lattices in room-temperature atoms. Our findings reveal the relationship between 1D topological matter and optical nonreciprocity, simplifying the design of topologically resilient nonreciprocal devices.
  arXiv  Detail & Related papers  (2024-09-26T06:12:07Z)
• Neural Message Passing Induced by Energy-Constrained Diffusion [79.9193447649011]
  We propose an energy-constrained diffusion model as a principled interpretable framework for understanding the mechanism of MPNNs. We show that the new model can yield promising performance for cases where the data structures are observed (as a graph), partially observed or completely unobserved.
  arXiv  Detail & Related papers  (2024-09-13T17:54:41Z)
• Accelerating Diffusion for SAR-to-Optical Image Translation via Adversarial Consistency Distillation [5.234109158596138]
  We propose a new training framework for SAR-to-optical image translation. Our method employs consistency distillation to reduce iterative inference steps and integrates adversarial learning to ensure image clarity and minimize color shifts. The results demonstrate that our approach significantly improves inference speed by 131 times while maintaining the visual quality of the generated images.
  arXiv  Detail & Related papers  (2024-07-08T16:36:12Z)
• Distilling Diffusion Models into Conditional GANs [90.76040478677609]
  We distill a complex multistep diffusion model into a single-step conditional GAN student model. For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space. We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
  arXiv  Detail & Related papers  (2024-05-09T17:59:40Z)
• Spatial super-resolution in nanosensing with blinking emitters [79.16635054977068]
  We propose a method of spatial resolution enhancement in metrology with blinking fluorescent nanosensors. We believe that blinking fluorescent sensing agents being complemented with the developed image analysis technique could be utilized routinely in the life science sector.
  arXiv  Detail & Related papers  (2024-02-27T10:38:05Z)
• Controlling Text-to-Image Diffusion by Orthogonal Finetuning [74.21549380288631]
  We introduce a principled finetuning method -- Orthogonal Finetuning (OFT) for adapting text-to-image diffusion models to downstream tasks. Unlike existing methods, OFT can provably preserve hyperspherical energy which characterizes the pairwise neuron relationship on the unit hypersphere. We empirically show that our OFT framework outperforms existing methods in generation quality and convergence speed.
  arXiv  Detail & Related papers  (2023-06-12T17:59:23Z)
• Super-resolution enhancement in bi-photon spatial mode demultiplexin [0.0]
  Imaging systems measuring intensity in the far field succumb to Rayleigh's curse, a resolution limitation dictated by the finite aperture of the optical system. Many proof-of-principle and some two-dimensional imaging experiments have shown that, by using spatial mode demultiplexing (SPADE), the field information collected is maximal.
  arXiv  Detail & Related papers  (2022-12-20T17:40:46Z)
• Manifold-aware Synthesis of High-resolution Diffusion from Structural Imaging [12.96280888284293]
  We propose a network architecture for the direct generation of diffusion tensors (DT) and diffusion orientation distribution functions (dODFs) from high-resolution T1w images. Our approach improves the fractional anisotropy mean squared error (FA MSE) between the synthesized diffusion and the ground-truth by more than 23%. While our method is able to generate high-resolution diffusion images from structural inputs in less than 15 seconds, we acknowledge and discuss the limits of diffusion inference solely relying on T1w images.
  arXiv  Detail & Related papers  (2021-08-09T16:00:40Z)
• Two-photon resonance fluorescence of two interacting non-identical quantum emitters [77.34726150561087]
  We study a system of two interacting, non-indentical quantum emitters driven by a coherent field. We show that the features imprinted by the two-photon dynamics into the spectrum of resonance fluorescence are particularly sensitive to changes in the distance between emitters. This can be exploited for applications such as superresolution imaging of point-like sources.
  arXiv  Detail & Related papers  (2021-06-04T16:13:01Z)
• Invertible Neural Networks for Uncertainty Quantification in Photoacoustic Imaging [22.690971184202944]
  In this work, we present a new approach for handling this specific type of uncertainty by leveraging the concept of conditional invertible neural networks (cINNs) Specifically, we propose going beyond commonly used point estimates for tissue oxygenation and converting single-pixel initial pressure spectra to the full posterior probability density. Based on the presented architecture, we demonstrate two use cases which leverage this information to not only detect and quantify but also to compensate for uncertainties.
  arXiv  Detail & Related papers  (2020-11-10T14:17:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.