Zero-Shot Scene Change Detection

• URL: http://arxiv.org/abs/2406.11210v1
• Date: Mon, 17 Jun 2024 05:03:44 GMT
• Title: Zero-Shot Scene Change Detection
• Authors: Kyusik Cho, Dong Yeop Kim, Euntai Kim,
• Abstract summary: Our method takes advantage of the change detection effect of the tracking model by inputting reference and query images instead of consecutive frames. We extend our approach to video to exploit rich temporal information, enhancing scene change detection performance.
• Score: 14.095215136905553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel, training-free approach to scene change detection. Our method leverages tracking models, which inherently perform change detection between consecutive frames of video by identifying common objects and detecting new or missing objects. Specifically, our method takes advantage of the change detection effect of the tracking model by inputting reference and query images instead of consecutive frames. Furthermore, we focus on the content gap and style gap between two input images in change detection, and address both issues by proposing adaptive content threshold and style bridging layers, respectively. Finally, we extend our approach to video to exploit rich temporal information, enhancing scene change detection performance. We compare our approach and baseline through various experiments. While existing train-based baseline tend to specialize only in the trained domain, our method shows consistent performance across various domains, proving the competitiveness of our approach.

Related papers

• ZeroSCD: Zero-Shot Street Scene Change Detection [2.3020018305241337]
  Scene Change Detection is a challenging task in computer vision and robotics. Traditional change detection methods rely on training models that take these image pairs as input and estimate the changes. We propose ZeroSCD, a zero-shot scene change detection framework that eliminates the need for training.
  arXiv  Detail & Related papers  (2024-09-23T17:53:44Z)
• Learning to search for and detect objects in foveal images using deep learning [3.655021726150368]
  This study employs a fixation prediction model that emulates human objective-guided attention of searching for a given class in an image. The foveated pictures at each fixation point are then classified to determine whether the target is present or absent in the scene. We present a novel dual task model capable of performing fixation prediction and detection simultaneously, allowing knowledge transfer between the two tasks.
  arXiv  Detail & Related papers  (2023-04-12T09:50:25Z)
• Learning Transformations To Reduce the Geometric Shift in Object Detection [60.20931827772482]
  We tackle geometric shifts emerging from variations in the image capture process. We introduce a self-training approach that learns a set of geometric transformations to minimize these shifts. We evaluate our method on two different shifts, i.e., a camera's field of view (FoV) change and a viewpoint change.
  arXiv  Detail & Related papers  (2023-01-13T11:55:30Z)
• Self-Pair: Synthesizing Changes from Single Source for Object Change Detection in Remote Sensing Imagery [6.586756080460231]
  We train a change detector using two spatially unrelated images with corresponding semantic labels such as building. We show that manipulating the source image as an after-image is crucial to the performance of change detection. Our method outperforms existing methods based on single-temporal supervision.
  arXiv  Detail & Related papers  (2022-12-20T13:26:42Z)
• Tracking by Associating Clips [110.08925274049409]
  In this paper, we investigate an alternative by treating object association as clip-wise matching. Our new perspective views a single long video sequence as multiple short clips, and then the tracking is performed both within and between the clips. The benefits of this new approach are two folds. First, our method is robust to tracking error accumulation or propagation, as the video chunking allows bypassing the interrupted frames. Second, the multiple frame information is aggregated during the clip-wise matching, resulting in a more accurate long-range track association than the current frame-wise matching.
  arXiv  Detail & Related papers  (2022-12-20T10:33:17Z)
• The Change You Want to See [91.3755431537592]
  Given two images of the same scene, being able to automatically detect the changes in them has practical applications in a variety of domains. We tackle the change detection problem with the goal of detecting "object-level" changes in an image pair despite differences in their viewpoint and illumination.
  arXiv  Detail & Related papers  (2022-09-28T18:10:09Z)
• dual unet:a novel siamese network for change detection with cascade differential fusion [4.651756476458979]
  We propose a novel Siamese neural network for change detection task, namely Dual-UNet. In contrast to previous individually encoded the bitemporal images, we design an encoder differential-attention module to focus on the spatial difference relationships of pixels. Experiments demonstrate that the proposed approach consistently outperforms the most advanced methods on popular seasonal change detection datasets.
  arXiv  Detail & Related papers  (2022-08-12T14:24:09Z)
• Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing [55.73722120043086]
  We introduce Contrast and Mix (CoMix), a new contrastive learning framework that aims to learn discriminative invariant feature representations for unsupervised video domain adaptation. First, we utilize temporal contrastive learning to bridge the domain gap by maximizing the similarity between encoded representations of an unlabeled video at two different speeds. Second, we propose a novel extension to the temporal contrastive loss by using background mixing that allows additional positives per anchor, thus adapting contrastive learning to leverage action semantics shared across both domains.
  arXiv  Detail & Related papers  (2021-10-28T14:03:29Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• City-scale Scene Change Detection using Point Clouds [71.73273007900717]
  We propose a method for detecting structural changes in a city using images captured from mounted cameras over two different times. A direct comparison of the two point clouds for change detection is not ideal due to inaccurate geo-location information. To circumvent this problem, we propose a deep learning-based non-rigid registration on the point clouds. Experiments show that our method is able to detect scene changes effectively, even in the presence of viewpoint and illumination differences.
  arXiv  Detail & Related papers  (2021-03-26T08:04:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.