Enhancing Scene Classification in Cloudy Image Scenarios: A Collaborative Transfer Method with Information Regulation Mechanism using Optical Cloud-Covered and SAR Remote Sensing Images

• URL: http://arxiv.org/abs/2501.04283v1
• Date: Wed, 08 Jan 2025 05:14:36 GMT
• Title: Enhancing Scene Classification in Cloudy Image Scenarios: A Collaborative Transfer Method with Information Regulation Mechanism using Optical Cloud-Covered and SAR Remote Sensing Images
• Authors: Yuze Wang, Rong Xiao, Haifeng Li, Mariana Belgiu, Chao Tao,
• Abstract summary: This study presents a scene classification transfer method that combines multi-modality data.<n>It aims to transfer the source domain model trained on cloudfree optical data to the target domain that includes both cloudy optical and SAR data at low cost.
• Score: 6.35948253619752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In remote sensing scene classification, leveraging the transfer methods with well-trained optical models is an efficient way to overcome label scarcity. However, cloud contamination leads to optical information loss and significant impacts on feature distribution, challenging the reliability and stability of transferred target models. Common solutions include cloud removal for optical data or directly using Synthetic aperture radar (SAR) data in the target domain. However, cloud removal requires substantial auxiliary data for support and pre-training, while directly using SAR disregards the unobstructed portions of optical data. This study presents a scene classification transfer method that synergistically combines multi-modality data, which aims to transfer the source domain model trained on cloudfree optical data to the target domain that includes both cloudy optical and SAR data at low cost. Specifically, the framework incorporates two parts: (1) the collaborative transfer strategy, based on knowledge distillation, enables the efficient prior knowledge transfer across heterogeneous data; (2) the information regulation mechanism (IRM) is proposed to address the modality imbalance issue during transfer. It employs auxiliary models to measure the contribution discrepancy of each modality, and automatically balances the information utilization of modalities during the target model learning process at the sample-level. The transfer experiments were conducted on simulated and real cloud datasets, demonstrating the superior performance of the proposed method compared to other solutions in cloud-covered scenarios. We also verified the importance and limitations of IRM, and further discussed and visualized the modality imbalance problem during the model transfer. Codes are available at https://github.com/wangyuze-csu/ESCCS

Related papers

• DehazeMamba: SAR-guided Optical Remote Sensing Image Dehazing with Adaptive State Space Model [27.83437788159158]
  We introduce DehazeMamba, a novel SAR-guided dehazing network built on a progressive haze decoupling fusion strategy. Our approach incorporates two key innovations: a Haze Perception and Decoupling Module (HPDM) that dynamically identifies haze-affected regions through optical-SAR difference analysis, and a Progressive Fusion Module (PFM) that mitigates domain shift through a two-stage fusion process based on feature quality assessment. Extensive experiments demonstrate that DehazeMamba significantly outperforms state-of-the-art methods, achieving a 0.73 dB improvement in PSNR and substantial enhancements in downstream tasks such as
  arXiv  Detail & Related papers  (2025-03-17T11:25:05Z)
• A Semantic-Loss Function Modeling Framework With Task-Oriented Machine Learning Perspectives [26.82506860792313]
  The performance of data-driven Earth Observation (EO) applications is heavily influenced by the data collection and transmission processes. Adopting the concepts of Semantic Communication (SC) offers a promising solution by prioritizing the transmission of essential data semantics over raw information. This work proposes a novel data-fitting framework to empirically model the semantic loss using real-world EO datasets and domain-specific insights.
  arXiv  Detail & Related papers  (2025-03-12T23:45:11Z)
• Synthetic Data is an Elegant GIFT for Continual Vision-Language Models [52.343627275005026]
  GIFT is a novel continual fine-tuning approach to overcome catastrophic forgetting in Vision-Language Models. We employ a pre-trained diffusion model to recreate both pre-training and learned downstream task data. Our method consistently outperforms previous state-of-the-art approaches across various settings.
  arXiv  Detail & Related papers  (2025-03-06T09:09:18Z)
• LPRnet: A self-supervised registration network for LiDAR and photogrammetric point clouds [38.42527849407057]
  LiDAR and photogrammetry are active and passive remote sensing techniques for point cloud acquisition, respectively. Due to the fundamental differences in sensing mechanisms, spatial distributions and coordinate systems, their point clouds exhibit significant discrepancies in density, precision, noise, and overlap. This paper proposes a self-supervised registration network based on a masked autoencoder, focusing on heterogeneous LiDAR and photogrammetric point clouds.
  arXiv  Detail & Related papers  (2025-01-10T02:36:37Z)
• HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks [7.06787067270941]
  The integration of hyperspectral imaging (HSI) and LiDAR data within new linear feature spaces offers a promising solution to the challenges posed by the high-dimensionality and redundancy inherent in HSIs.<n>This study introduces a dual linear fused space framework that capitalizes on bidirectional reversed convolutional neural network (CNN) pathways, coupled with a specialized spatial analysis block.<n>The proposed method not only enhances data processing and classification accuracy, but also mitigates the computational burden typically associated with advanced models such as Transformers.
  arXiv  Detail & Related papers  (2024-11-30T01:08:08Z)
• Enhancing Information Maximization with Distance-Aware Contrastive Learning for Source-Free Cross-Domain Few-Shot Learning [55.715623885418815]
  Cross-Domain Few-Shot Learning methods require access to source domain data to train a model in the pre-training phase. Due to increasing concerns about data privacy and the desire to reduce data transmission and training costs, it is necessary to develop a CDFSL solution without accessing source data. This paper proposes an Enhanced Information Maximization with Distance-Aware Contrastive Learning method to address these challenges.
  arXiv  Detail & Related papers  (2024-03-04T12:10:24Z)
• Diffusion Enhancement for Cloud Removal in Ultra-Resolution Remote Sensing Imagery [48.14610248492785]
  Cloud layers severely compromise the quality and effectiveness of optical remote sensing (RS) images. Existing deep-learning (DL)-based Cloud Removal (CR) techniques encounter difficulties in accurately reconstructing the original visual authenticity and detailed semantic content of the images. This work proposes enhancements at the data and methodology fronts to tackle this challenge.
  arXiv  Detail & Related papers  (2024-01-25T13:14:17Z)
• Erasing Undesirable Influence in Diffusion Models [51.225365010401006]
  Diffusion models are highly effective at generating high-quality images but pose risks, such as the unintentional generation of NSFW (not safe for work) content. In this work, we introduce EraseDiff, an algorithm designed to preserve the utility of the diffusion model on retained data while removing the unwanted information associated with the data to be forgotten.
  arXiv  Detail & Related papers  (2024-01-11T09:30:36Z)
• Imposing Consistency for Optical Flow Estimation [73.53204596544472]
  Imposing consistency through proxy tasks has been shown to enhance data-driven learning. This paper introduces novel and effective consistency strategies for optical flow estimation.
  arXiv  Detail & Related papers  (2022-04-14T22:58:30Z)
• Probing transfer learning with a model of synthetic correlated datasets [11.53207294639557]
  Transfer learning can significantly improve the sample efficiency of neural networks. We re-think a solvable model of synthetic data as a framework for modeling correlation between data-sets. We show that our model can capture a range of salient features of transfer learning with real data.
  arXiv  Detail & Related papers  (2021-06-09T22:15:41Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso [102.84915019938413]
  Non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG) offer promise of non-invasive techniques. The problem of source localization, or source imaging, poses however a high-dimensional statistical inference challenge. We propose an ensemble of desparsified multi-task Lasso (ecd-MTLasso) to deal with this problem.
  arXiv  Detail & Related papers  (2020-09-29T21:17:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.