Segment Any Change
- URL: http://arxiv.org/abs/2402.01188v2
- Date: Thu, 15 Feb 2024 04:07:46 GMT
- Title: Segment Any Change
- Authors: Zhuo Zheng, Yanfei Zhong, Liangpei Zhang, Stefano Ermon
- Abstract summary: We propose a new type of change detection model that supports zero-shot prediction and generalization on unseen change types and data distributions.
AnyChange is built on the segment anything model (SAM) via our training-free adaptation method, bitemporal latent matching.
We also propose a point query mechanism to enable AnyChange's zero-shot object-centric change detection capability.
- Score: 70.17716393332482
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Visual foundation models have achieved remarkable results in zero-shot image
classification and segmentation, but zero-shot change detection remains an open
problem. In this paper, we propose the segment any change models (AnyChange), a
new type of change detection model that supports zero-shot prediction and
generalization on unseen change types and data distributions. AnyChange is
built on the segment anything model (SAM) via our training-free adaptation
method, bitemporal latent matching. By revealing and exploiting intra-image and
inter-image semantic similarities in SAM's latent space, bitemporal latent
matching endows SAM with zero-shot change detection capabilities in a
training-free way. We also propose a point query mechanism to enable
AnyChange's zero-shot object-centric change detection capability. We perform
extensive experiments to confirm the effectiveness of AnyChange for zero-shot
change detection. AnyChange sets a new record on the SECOND benchmark for
unsupervised change detection, exceeding the previous SOTA by up to 4.4% F$_1$
score, and achieving comparable accuracy with negligible manual annotations (1
pixel per image) for supervised change detection.
Related papers
- Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model [62.337749660637755]
We present change data generators based on generative models which are cheap and automatic.
Changen2 is a generative change foundation model that can be trained at scale via self-supervision.
The resulting model possesses inherent zero-shot change detection capabilities and excellent transferability.
arXiv Detail & Related papers (2024-06-26T01:03:39Z) - Single-Temporal Supervised Learning for Universal Remote Sensing Change Detection [21.622442722863028]
We propose single-temporal supervised learning (STAR) for universal remote sensing change detection.
Star enables us to train a high-accuracy change detector only using unpaired labeled images.
ChangeStar2 achieves state-of-the-art performances on eight public remote sensing change detection datasets.
arXiv Detail & Related papers (2024-06-22T00:03:21Z) - Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysis [28.3763053922823]
Current RSICI technology encompasses change detection and change captioning, each with its limitations in providing comprehensive interpretation.
We propose an interactive Change-Agent, which can follow user instructions to achieve comprehensive change interpretation.
The Change-Agent integrates a multi-level change interpretation (MCI) model as the eyes and a large language model (LLM) as the brain.
arXiv Detail & Related papers (2024-03-28T17:55:42Z) - BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model [65.92173280096588]
We address the challenge of image resolution variation for the Segment Anything Model (SAM)
SAM, known for its zero-shot generalizability, exhibits a performance degradation when faced with datasets with varying image sizes.
We present a bias-mode attention mask that allows each token to prioritize neighboring information.
arXiv Detail & Related papers (2024-01-04T15:34:44Z) - MS-Former: Memory-Supported Transformer for Weakly Supervised Change
Detection with Patch-Level Annotations [50.79913333804232]
We propose a memory-supported transformer (MS-Former) for weakly supervised change detection.
MS-Former consists of a bi-directional attention block (BAB) and a patch-level supervision scheme (PSS)
Experimental results on three benchmark datasets demonstrate the effectiveness of our proposed method in the change detection task.
arXiv Detail & Related papers (2023-11-16T09:57:29Z) - Self-Pair: Synthesizing Changes from Single Source for Object Change
Detection in Remote Sensing Imagery [6.586756080460231]
We train a change detector using two spatially unrelated images with corresponding semantic labels such as building.
We show that manipulating the source image as an after-image is crucial to the performance of change detection.
Our method outperforms existing methods based on single-temporal supervision.
arXiv Detail & Related papers (2022-12-20T13:26:42Z) - The Change You Want to See [91.3755431537592]
Given two images of the same scene, being able to automatically detect the changes in them has practical applications in a variety of domains.
We tackle the change detection problem with the goal of detecting "object-level" changes in an image pair despite differences in their viewpoint and illumination.
arXiv Detail & Related papers (2022-09-28T18:10:09Z) - Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries.
This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management.
We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
arXiv Detail & Related papers (2020-10-12T13:26:30Z) - DASNet: Dual attentive fully convolutional siamese networks for change
detection of high resolution satellite images [17.839181739760676]
The research objective is to identity the change information of interest and filter out the irrelevant change information as interference factors.
Recently, the rise of deep learning has provided new tools for change detection, which have yielded impressive results.
We propose a new method, namely, dual attentive fully convolutional Siamese networks (DASNet) for change detection in high-resolution images.
arXiv Detail & Related papers (2020-03-07T16:57:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.