SyntStereo2Real: Edge-Aware GAN for Remote Sensing Image-to-Image Translation while Maintaining Stereo Constraint

• URL: http://arxiv.org/abs/2404.09277v1
• Date: Sun, 14 Apr 2024 14:58:52 GMT
• Title: SyntStereo2Real: Edge-Aware GAN for Remote Sensing Image-to-Image Translation while Maintaining Stereo Constraint
• Authors: Vasudha Venkatesan, Daniel Panangian, Mario Fuentes Reyes, Ksenia Bittner,
• Abstract summary: Current methods involve combining two networks, an unpaired image-to-image translation network and a stereo-matching network. We propose an edge-aware GAN-based network that effectively tackles both tasks simultaneously. We demonstrate that our model produces qualitatively and quantitatively superior results than existing models, and its applicability extends to diverse domains.
• Score: 1.8749305679160366
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the field of remote sensing, the scarcity of stereo-matched and particularly lack of accurate ground truth data often hinders the training of deep neural networks. The use of synthetically generated images as an alternative, alleviates this problem but suffers from the problem of domain generalization. Unifying the capabilities of image-to-image translation and stereo-matching presents an effective solution to address the issue of domain generalization. Current methods involve combining two networks, an unpaired image-to-image translation network and a stereo-matching network, while jointly optimizing them. We propose an edge-aware GAN-based network that effectively tackles both tasks simultaneously. We obtain edge maps of input images from the Sobel operator and use it as an additional input to the encoder in the generator to enforce geometric consistency during translation. We additionally include a warping loss calculated from the translated images to maintain the stereo consistency. We demonstrate that our model produces qualitatively and quantitatively superior results than existing models, and its applicability extends to diverse domains, including autonomous driving.

Related papers

• SCONE-GAN: Semantic Contrastive learning-based Generative Adversarial Network for an end-to-end image translation [18.93434486338439]
  SCONE-GAN is shown to be effective for learning to generate realistic and diverse scenery images. For more realistic and diverse image generation we introduce style reference image. We validate the proposed algorithm for image-to-image translation and stylizing outdoor images.
  arXiv  Detail & Related papers  (2023-11-07T10:29:16Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• Wavelet-based Unsupervised Label-to-Image Translation [9.339522647331334]
  We propose a new Unsupervised paradigm for SIS (USIS) that makes use of a self-supervised segmentation loss and whole image wavelet based discrimination. We test our methodology on 3 challenging datasets and demonstrate its ability to bridge the performance gap between paired and unpaired models.
  arXiv  Detail & Related papers  (2023-05-16T17:48:44Z)
• Multi-cropping Contrastive Learning and Domain Consistency for Unsupervised Image-to-Image Translation [5.562419999563734]
  We propose a novel unsupervised image-to-image translation framework based on multi-cropping contrastive learning and domain consistency, called MCDUT. In many image-to-image translation tasks, our method achieves state-of-the-art results, and the advantages of our method have been proven through comparison experiments and ablation research.
  arXiv  Detail & Related papers  (2023-04-24T16:20:28Z)
• Global and Local Alignment Networks for Unpaired Image-to-Image Translation [170.08142745705575]
  The goal of unpaired image-to-image translation is to produce an output image reflecting the target domain's style. Due to the lack of attention to the content change in existing methods, semantic information from source images suffers from degradation during translation. We introduce a novel approach, Global and Local Alignment Networks (GLA-Net) Our method effectively generates sharper and more realistic images than existing approaches.
  arXiv  Detail & Related papers  (2021-11-19T18:01:54Z)
• Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network [92.01145655155374]
  We present an unsupervised image enhancement generative network (UEGAN) It learns the corresponding image-to-image mapping from a set of images with desired characteristics in an unsupervised manner. Results show that the proposed model effectively improves the aesthetic quality of images.
  arXiv  Detail & Related papers  (2020-12-30T03:22:46Z)
• StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching [56.95846963856928]
  Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias. We propose an end-to-end training framework with domain translation and stereo matching networks to tackle this challenge.
  arXiv  Detail & Related papers  (2020-05-05T03:11:38Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.