Semantic-aware Network for Aerial-to-Ground Image Synthesis

• URL: http://arxiv.org/abs/2308.06945v1
• Date: Mon, 14 Aug 2023 05:37:07 GMT
• Title: Semantic-aware Network for Aerial-to-Ground Image Synthesis
• Authors: Jinhyun Jang, Taeyong Song, Kwanghoon Sohn
• Abstract summary: We propose a novel framework to explore the challenges by imposing enhanced structural alignment and semantic awareness. We introduce a novel semantic-attentive feature transformation module that allows to reconstruct the complex geographic structures. We also propose semantic-aware loss functions by leveraging a pre-trained segmentation network.
• Score: 42.360670351361584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aerial-to-ground image synthesis is an emerging and challenging problem that aims to synthesize a ground image from an aerial image. Due to the highly different layout and object representation between the aerial and ground images, existing approaches usually fail to transfer the components of the aerial scene into the ground scene. In this paper, we propose a novel framework to explore the challenges by imposing enhanced structural alignment and semantic awareness. We introduce a novel semantic-attentive feature transformation module that allows to reconstruct the complex geographic structures by aligning the aerial feature to the ground layout. Furthermore, we propose semantic-aware loss functions by leveraging a pre-trained segmentation network. The network is enforced to synthesize realistic objects across various classes by separately calculating losses for different classes and balancing them. Extensive experiments including comparisons with previous methods and ablation studies show the effectiveness of the proposed framework both qualitatively and quantitatively.

Related papers

• Reconciling Semantic Controllability and Diversity for Remote Sensing Image Synthesis with Hybrid Semantic Embedding [12.330893658398042]
  We present a Hybrid Semantic Embedding Guided Geneversarative Adversarial Network (HySEGGAN) for controllable and efficient remote sensing image synthesis. Motivated by feature description, we propose a hybrid semantic Embedding method, that coordinates fine-grained local semantic layouts. A Semantic Refinement Network (SRN) is introduced, incorporating a novel loss function to ensure fine-grained semantic feedback.
  arXiv  Detail & Related papers  (2024-11-22T07:51:36Z)
• Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery [51.73680703579997]
  We present a neural radiance field method for urban-scale semantic and building-level instance segmentation from aerial images. objects in urban aerial images exhibit substantial variations in size, including buildings, cars, and roads. We introduce a scale-adaptive semantic label fusion strategy that enhances the segmentation of objects of varying sizes. We then introduce a novel cross-view instance label grouping strategy to mitigate the multi-view inconsistency problem in the 2D instance labels.
  arXiv  Detail & Related papers  (2024-03-18T14:15:39Z)
• Layered Rendering Diffusion Model for Zero-Shot Guided Image Synthesis [60.260724486834164]
  This paper introduces innovative solutions to enhance spatial controllability in diffusion models reliant on text queries. We present two key innovations: Vision Guidance and the Layered Rendering Diffusion framework. We apply our method to three practical applications: bounding box-to-image, semantic mask-to-image and image editing.
  arXiv  Detail & Related papers  (2023-11-30T10:36:19Z)
• Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis [139.2216271759332]
  We propose a novel ECGAN for the challenging semantic image synthesis task. The semantic labels do not provide detailed structural information, making it challenging to synthesize local details and structures. The widely adopted CNN operations such as convolution, down-sampling, and normalization usually cause spatial resolution loss. We propose a novel contrastive learning method, which aims to enforce pixel embeddings belonging to the same semantic class to generate more similar image content.
  arXiv  Detail & Related papers  (2023-07-22T14:17:19Z)
• Uni-Removal: A Semi-Supervised Framework for Simultaneously Addressing Multiple Degradations in Real-World Images [6.3351090376024155]
  Uni-Removal is a twostage semi-supervised framework for addressing the removal of multiple degradations in real-world images. In the knowledge transfer stage, Uni-Removal leverages a supervised multi-teacher and student architecture. In the domain adaptation stage, unsupervised fine-tuning is performed by incorporating an adversarial discriminator on real-world images.
  arXiv  Detail & Related papers  (2023-07-11T07:18:15Z)
• Enhancing Photorealism Enhancement [83.88433283714461]
  We present an approach to enhancing the realism of synthetic images using a convolutional network. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We report substantial gains in stability and realism in comparison to recent image-to-image translation methods.
  arXiv  Detail & Related papers  (2021-05-10T19:00:49Z)
• Adversarial Image Composition with Auxiliary Illumination [53.89445873577062]
  We propose an Adversarial Image Composition Net (AIC-Net) that achieves realistic image composition. A novel branched generation mechanism is proposed, which disentangles the generation of shadows and the transfer of foreground styles. Experiments on pedestrian and car composition tasks show that the proposed AIC-Net achieves superior composition performance.
  arXiv  Detail & Related papers  (2020-09-17T12:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.