Band Prompting Aided SAR and Multi-Spectral Data Fusion Framework for Local Climate Zone Classification

• URL: http://arxiv.org/abs/2412.18235v1
• Date: Tue, 24 Dec 2024 07:40:07 GMT
• Title: Band Prompting Aided SAR and Multi-Spectral Data Fusion Framework for Local Climate Zone Classification
• Authors: Haiyan Lan, Shujun Li, Mingjie Xie, Xuanjia Zhao, Hongning Liu, Pengming Feng, Dongli Xu, Guangjun He, Jian Guan,
• Abstract summary: Local climate zone (LCZ) classification is of great value for understanding the complex interactions between urban development and local climate. Recent studies have increasingly focused on the fusion of synthetic aperture radar (SAR) and multi-spectral data to improve LCZ classification performance. In this paper, a novel band prompting aided data fusion framework is proposed for LCZ classification, namely BP-LCZ. The experimental results demonstrate the effectiveness and superiority of the proposed data fusion framework.
• Score: 20.71392764471532
• License:
• Abstract: Local climate zone (LCZ) classification is of great value for understanding the complex interactions between urban development and local climate. Recent studies have increasingly focused on the fusion of synthetic aperture radar (SAR) and multi-spectral data to improve LCZ classification performance. However, it remains challenging due to the distinct physical properties of these two types of data and the absence of effective fusion guidance. In this paper, a novel band prompting aided data fusion framework is proposed for LCZ classification, namely BP-LCZ, which utilizes textual prompts associated with band groups to guide the model in learning the physical attributes of different bands and semantics of various categories inherent in SAR and multi-spectral data to augment the fused feature, thus enhancing LCZ classification performance. Specifically, a band group prompting (BGP) strategy is introduced to align the visual representation effectively at the level of band groups, which also facilitates a more adequate extraction of semantic information of different bands with textual information. In addition, a multivariate supervised matrix (MSM) based training strategy is proposed to alleviate the problem of positive and negative sample confusion by completing the supervised information. The experimental results demonstrate the effectiveness and superiority of the proposed data fusion framework.

Related papers

• A Comprehensive Survey on Spectral Clustering with Graph Structure Learning [10.579153358536372]
  Spectral clustering is a powerful technique for clustering high-dimensional data. We explore various graph clustering techniques, including pairwise, anchor, and hypergraph-based methods. We discuss multi-view clustering frameworks, examining their applications within one-step and two-step clustering processes.
  arXiv  Detail & Related papers  (2025-01-23T12:06:32Z)
• PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model [76.95536611263356]
  PolSAR data presents unique challenges due to its rich and complex characteristics. Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used. Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively. We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy.
  arXiv  Detail & Related papers  (2024-12-17T09:59:53Z)
• An Enhanced Classification Method Based on Adaptive Multi-Scale Fusion for Long-tailed Multispectral Point Clouds [67.96583737413296]
  We propose an enhanced classification method based on adaptive multi-scale fusion for MPCs with long-tailed distributions. In the training set generation stage, a grid-balanced sampling strategy is designed to reliably generate training samples from sparse labeled datasets. In the feature learning stage, a multi-scale feature fusion module is proposed to fuse shallow features of land-covers at different scales.
  arXiv  Detail & Related papers  (2024-12-16T03:21:20Z)
• Revisiting Self-Supervised Heterogeneous Graph Learning from Spectral Clustering Perspective [52.662463893268225]
  Self-supervised heterogeneous graph learning (SHGL) has shown promising potential in diverse scenarios. Existing SHGL methods encounter two significant limitations. We introduce a novel framework enhanced by rank and dual consistency constraints.
  arXiv  Detail & Related papers  (2024-12-01T09:33:20Z)
• LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition [9.103869144049014]
  LiGAR is a LIDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition. Our framework incorporates a Multi-Scale LIDAR Transformer, Cross-Modal Guided Attention, and an Adaptive Fusion Module. LiGAR's hierarchical architecture captures group activities at various granularities, from individual actions to scene-level dynamics.
  arXiv  Detail & Related papers  (2024-10-28T15:11:49Z)
• A Framework for Fine-Tuning LLMs using Heterogeneous Feedback [69.51729152929413]
  We present a framework for fine-tuning large language models (LLMs) using heterogeneous feedback. First, we combine the heterogeneous feedback data into a single supervision format, compatible with methods like SFT and RLHF. Next, given this unified feedback dataset, we extract a high-quality and diverse subset to obtain performance increases.
  arXiv  Detail & Related papers  (2024-08-05T23:20:32Z)
• Unsupervised Band Selection Using Fused HSI and LiDAR Attention Integrating With Autoencoder [16.742768644585684]
  Band selection in hyperspectral imaging (HSI) is critical for optimising data processing and enhancing analytical accuracy. Traditional approaches have predominantly concentrated on analysing spectral and pixel characteristics within individual bands independently. This paper introduces a novel unsupervised band selection framework that incorporates attention mechanisms and an Autoencoder for reconstruction-based band selection.
  arXiv  Detail & Related papers  (2024-04-08T07:47:28Z)
• LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Image Classification [16.742768644585684]
  This paper introduces a cross-attention mechanism for the selection of HSI bands guided by LiDAR data. The method ensures that the selected HSI bands drastically reduce redundancy and computational requirements. Experiments have been undertaken on three paired HSI and LiDAR data sets: Houston 2013, Trento and MUUFL.
  arXiv  Detail & Related papers  (2024-04-05T04:11:31Z)
• DiffSpectralNet : Unveiling the Potential of Diffusion Models for Hyperspectral Image Classification [6.521187080027966]
  We propose a new network called DiffSpectralNet, which combines diffusion and transformer techniques. First, we use an unsupervised learning framework based on the diffusion model to extract both high-level and low-level spectral-spatial features. The diffusion method is capable of extracting diverse and meaningful spectral-spatial features, leading to improvement in HSI classification.
  arXiv  Detail & Related papers  (2023-10-29T15:26:37Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning [77.72330187258498]
  We propose a novel Entropy-guided Reinforced Partial Convolutional Network (ERPCNet) ERPCNet extracts and aggregates localities based on semantic relevance and visual correlations without human-annotated regions. It not only discovers global-cooperative localities dynamically but also converges faster for policy gradient optimization.
  arXiv  Detail & Related papers  (2021-11-03T11:13:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.