Fine-grained building roof instance segmentation based on domain adapted pretraining and composite dual-backbone

• URL: http://arxiv.org/abs/2308.05358v1
• Date: Thu, 10 Aug 2023 05:54:57 GMT
• Title: Fine-grained building roof instance segmentation based on domain adapted pretraining and composite dual-backbone
• Authors: Guozhang Liu, Baochai Peng, Ting Liu, Pan Zhang, Mengke Yuan, Chaoran Lu, Ningning Cao, Sen Zhang, Simin Huang, Tao Wang
• Abstract summary: We propose a framework to fulfill semantic interpretation of individual buildings with high-resolution optical satellite imagery. Specifically, the leveraged domain adapted pretraining strategy and composite dual-backbone greatly facilitates the discnative feature learning. Experiment results show that our approach ranks in the first place of the 2023 IEEE GRSS Data Fusion Contest.
• Score: 13.09940764764909
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The diversity of building architecture styles of global cities situated on various landforms, the degraded optical imagery affected by clouds and shadows, and the significant inter-class imbalance of roof types pose challenges for designing a robust and accurate building roof instance segmentor. To address these issues, we propose an effective framework to fulfill semantic interpretation of individual buildings with high-resolution optical satellite imagery. Specifically, the leveraged domain adapted pretraining strategy and composite dual-backbone greatly facilitates the discriminative feature learning. Moreover, new data augmentation pipeline, stochastic weight averaging (SWA) training and instance segmentation based model ensemble in testing are utilized to acquire additional performance boost. Experiment results show that our approach ranks in the first place of the 2023 IEEE GRSS Data Fusion Contest (DFC) Track 1 test phase ($mAP_{50}$:50.6\%). Note-worthily, we have also explored the potential of multimodal data fusion with both optical satellite imagery and SAR data.

Related papers

• A Framework for Fine-Tuning LLMs using Heterogeneous Feedback [69.51729152929413]
  We present a framework for fine-tuning large language models (LLMs) using heterogeneous feedback. First, we combine the heterogeneous feedback data into a single supervision format, compatible with methods like SFT and RLHF. Next, given this unified feedback dataset, we extract a high-quality and diverse subset to obtain performance increases.
  arXiv  Detail & Related papers  (2024-08-05T23:20:32Z)
• UB-FineNet: Urban Building Fine-grained Classification Network for Open-access Satellite Images [7.435848987082052]
  We propose a deep network approach to fine-grained classification of urban buildings using open-access satellite images. A new fine-grained classification network with Category Information Balancing Module (CIBM) and Contrastive Supervision (CS) technique is proposed to mitigate the problem of class imbalance. Experiments on Hong Kong data set with 11 fine building types revealed promising classification results with a mean Top-1 accuracy of 60.45%.
  arXiv  Detail & Related papers  (2024-03-04T15:40:31Z)
• Multi-task deep learning for large-scale building detail extraction from high-resolution satellite imagery [13.544826927121992]
  Multi-task Building Refiner (MT-BR) is an adaptable neural network tailored for simultaneous extraction of building details from satellite imagery. For large-scale applications, we devise a novel spatial sampling scheme that strategically selects limited but representative image samples. MT-BR consistently outperforms other state-of-the-art methods in extracting building details across various metrics.
  arXiv  Detail & Related papers  (2023-10-29T04:43:30Z)
• Segment Any Building [8.12405696290333]
  This manuscript accentuates the potency of harnessing diversified datasets in tandem with cutting-edge representation learning paradigms for building segmentation in such images. Our avant-garde joint training regimen underscores the merit of our approach, bearing significant implications in pivotal domains such as urban infrastructural development, disaster mitigation strategies, and ecological surveillance. The outcomes of this research both fortify the foundations for ensuing scholarly pursuits and presage a horizon replete with innovative applications in the discipline of building segmentation.
  arXiv  Detail & Related papers  (2023-10-02T12:49:20Z)
• Bilevel Fast Scene Adaptation for Low-Light Image Enhancement [50.639332885989255]
  Enhancing images in low-light scenes is a challenging but widely concerned task in the computer vision. Main obstacle lies in the modeling conundrum from distribution discrepancy across different scenes. We introduce the bilevel paradigm to model the above latent correspondence. A bilevel learning framework is constructed to endow the scene-irrelevant generality of the encoder towards diverse scenes.
  arXiv  Detail & Related papers  (2023-06-02T08:16:21Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• A diverse large-scale building dataset and a novel plug-and-play domain generalization method for building extraction [2.578242050187029]
  We introduce a new building dataset and propose a novel domain generalization method to facilitate the development of building extraction from remote sensing images. The WHU-Mix building dataset consists of a training/validation set containing 43,727 diverse images collected from all over the world, and a test set containing 8402 images from five other cities on five continents. To further improve the generalization ability of a building extraction model, we propose a domain generalization method named batch style mixing (BSM)
  arXiv  Detail & Related papers  (2022-08-22T01:43:13Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Buildings Classification using Very High Resolution Satellite Imagery [0.769672852567215]
  We focus on buildings damage assessment (BDA) and buildings type classification (BTC) of residential and non-residential buildings. We propose a 2-stage deep learning-based approach, where first, buildings' footprints are extracted using a semantic segmentation model. We validate the proposed approach on two applications showing excellent accuracy and F1-score metrics.
  arXiv  Detail & Related papers  (2021-11-29T16:07:04Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• One-Shot Object Detection without Fine-Tuning [62.39210447209698]
  We introduce a two-stage model consisting of a first stage Matching-FCOS network and a second stage Structure-Aware Relation Module. We also propose novel training strategies that effectively improve detection performance. Our method exceeds the state-of-the-art one-shot performance consistently on multiple datasets.
  arXiv  Detail & Related papers  (2020-05-08T01:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.