A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detection

• URL: http://arxiv.org/abs/2312.01163v2
• Date: Sun, 11 Feb 2024 06:09:16 GMT
• Title: A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detection
• Authors: Kaiyu Li, Xiangyong Cao, Deyu Meng
• Abstract summary: Change detection (CD) is a critical task to observe and analyze dynamic processes of land cover. We propose a Bi-Temporal Adapter Network (BAN), which is a universal foundation model-based CD adaptation framework.
• Score: 54.01158175996638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Change detection (CD) is a critical task to observe and analyze dynamic processes of land cover. Although numerous deep learning-based CD models have performed excellently, their further performance improvements are constrained by the limited knowledge extracted from the given labelled data. On the other hand, the foundation models that emerged recently contain a huge amount of knowledge by scaling up across data modalities and proxy tasks. In this paper, we propose a Bi-Temporal Adapter Network (BAN), which is a universal foundation model-based CD adaptation framework aiming to extract the knowledge of foundation models for CD. The proposed BAN contains three parts, i.e. frozen foundation model (e.g., CLIP), bi-temporal adapter branch (Bi-TAB), and bridging modules between them. Specifically, BAN extracts general features through a frozen foundation model, which are then selected, aligned, and injected into Bi-TAB via the bridging modules. Bi-TAB is designed as a model-agnostic concept to extract task/domain-specific features, which can be either an existing arbitrary CD model or some hand-crafted stacked blocks. Beyond current customized models, BAN is the first extensive attempt to adapt the foundation model to the CD task. Experimental results show the effectiveness of our BAN in improving the performance of existing CD methods (e.g., up to 4.08\% IoU improvement) with only a few additional learnable parameters. More importantly, these successful practices show us the potential of foundation models for remote sensing CD. The code is available at \url{https://github.com/likyoo/BAN} and will be supported in our Open-CD.

Related papers

• High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study [64.06777376676513]
  We develop a few-shot segmentation (FSS) framework based on foundation models. To be specific, we propose a simple approach to extract implicit knowledge from foundation models to construct coarse correspondence. Experiments on two widely used datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-10T08:04:11Z)
• ChangeAnywhere: Sample Generation for Remote Sensing Change Detection via Semantic Latent Diffusion Model [4.677012401985776]
  ChangeAnywhere is a novel CD sample generation method using the semantic latent diffusion model and single-temporal images. ChangeAnywhere captures the two essentials of CD samples, i.e. change implies semantically different, and non-change implies reasonable change under the same semantic constraints. The ChangeAnywhere-100K significantly improved both zero-shot and few-shot performance on two CD benchmark datasets for various deep learning-based CD models.
  arXiv  Detail & Related papers  (2024-04-13T03:46:35Z)
• Cross-Domain Few-Shot Object Detection via Enhanced Open-Set Object Detector [72.05791402494727]
  This paper studies the challenging cross-domain few-shot object detection (CD-FSOD) It aims to develop an accurate object detector for novel domains with minimal labeled examples.
  arXiv  Detail & Related papers  (2024-02-05T15:25:32Z)
• An Exploratory Study on Automatic Identification of Assumptions in the Development of Deep Learning Frameworks [3.457512613793633]
  Existing approaches and tools for assumption management usually depend on manual identification of assumptions. This study intends to evaluate different classification models for the purpose of identification with respect to assumptions from the point of view of developers and users.
  arXiv  Detail & Related papers  (2024-01-08T03:50:03Z)
• TaCA: Upgrading Your Visual Foundation Model with Task-agnostic Compatible Adapter [21.41170708560114]
  A growing number of applications based on visual foundation models are emerging. In situations involving system upgrades, it becomes essential to re-train all downstream modules to adapt to the new foundation model. We introduce a parameter-efficient and task-agnostic adapter, dubbed TaCA, that facilitates compatibility across distinct foundation models.
  arXiv  Detail & Related papers  (2023-06-22T03:00:24Z)
• Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
  We make empirical studies of state-of-the-art UniDA methods using foundation models. We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models. Although simple, our method outperforms previous approaches in most benchmark tasks.
  arXiv  Detail & Related papers  (2023-05-18T16:28:29Z)
• PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations [1.2031796234206138]
  We propose PADA (Pruning Assisted Domain Adaptation) and zero out redundant weights from models pre-trained on large amounts of out-of-domain (OOD) data. The redundant weights can be identified through various pruning strategies which have been discussed in detail as a part of this work. Specifically, we investigate the effect of the recently discovered Task-Agnostic and Task-Aware pruning on PADA and propose a new pruning paradigm based on the latter. Our proposed CD-TAW methodology achieves up to 20.6% relative WER improvement over our baseline when fine-tuned on a 2-hour subset of Switch
  arXiv  Detail & Related papers  (2022-03-31T11:34:58Z)
• Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data [95.47859525676246]
  A recent study finds that existing few-shot learning methods, trained on the source domain, fail to generalize to the novel target domain when a domain gap is observed. In this paper, we realize that the labeled target data in Cross-Domain Few-Shot Learning has not been leveraged in any way to help the learning process.
  arXiv  Detail & Related papers  (2021-07-26T06:15:45Z)
• DGSAC: Density Guided Sampling and Consensus [4.808421423598809]
  Kernel Residual Density is a key differentiator between inliers and outliers. We propose two model selection algorithms, an optimal quadratic program based, and a greedy. We evaluate our method on a wide variety of tasks like planar segmentation, motion segmentation, vanishing point estimation, plane fitting to 3D point cloud, line, and circle fitting.
  arXiv  Detail & Related papers  (2020-06-03T17:42:53Z)
• Model Reuse with Reduced Kernel Mean Embedding Specification [70.044322798187]
  We present a two-phase framework for finding helpful models for a current application. In the upload phase, when a model is uploading into the pool, we construct a reduced kernel mean embedding (RKME) as a specification for the model. Then in the deployment phase, the relatedness of the current task and pre-trained models will be measured based on the value of the RKME specification.
  arXiv  Detail & Related papers  (2020-01-20T15:15:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.