Multi-View Pose-Agnostic Change Localization with Zero Labels

• URL: http://arxiv.org/abs/2412.03911v1
• Date: Thu, 05 Dec 2024 06:28:54 GMT
• Title: Multi-View Pose-Agnostic Change Localization with Zero Labels
• Authors: Chamuditha Jayanga Galappaththige, Jason Lai, Lloyd Windrim, Donald Dansereau, Niko Suenderhauf, Dimity Miller,
• Abstract summary: We propose a label-free, pose-agnostic change detection method that integrates information from multiple viewpoints. With as few as 5 images of the post-change scene, our approach can learn additional change channels in a 3DGS. Our change-aware 3D scene representation additionally enables the generation of accurate change masks for unseen viewpoints.
• Score: 4.997375878454274
• License:
• Abstract: Autonomous agents often require accurate methods for detecting and localizing changes in their environment, particularly when observations are captured from unconstrained and inconsistent viewpoints. We propose a novel label-free, pose-agnostic change detection method that integrates information from multiple viewpoints to construct a change-aware 3D Gaussian Splatting (3DGS) representation of the scene. With as few as 5 images of the post-change scene, our approach can learn additional change channels in a 3DGS and produce change masks that outperform single-view techniques. Our change-aware 3D scene representation additionally enables the generation of accurate change masks for unseen viewpoints. Experimental results demonstrate state-of-the-art performance in complex multi-object scenes, achieving a 1.7$\times$ and 1.6$\times$ improvement in Mean Intersection Over Union and F1 score respectively over other baselines. We also contribute a new real-world dataset to benchmark change detection in diverse challenging scenes in the presence of lighting variations.

Related papers

• Decoupling Appearance Variations with 3D Consistent Features in Gaussian Splatting [50.98884579463359]
  We propose DAVIGS, a method that decouples appearance variations in a plug-and-play manner. By transforming the rendering results at the image level instead of the Gaussian level, our approach can model appearance variations with minimal optimization time and memory overhead. We validate our method on several appearance-variant scenes, and demonstrate that it achieves state-of-the-art rendering quality with minimal training time and memory usage.
  arXiv  Detail & Related papers  (2025-01-18T14:55:58Z)
• Improving Zero-Shot Object-Level Change Detection by Incorporating Visual Correspondence [13.479857959236345]
  Existing change-detection approaches suffer from three major limitations. We introduce a novel method that leverages change correspondences during training to improve change detection accuracy. We are also the first to predict correspondences between pairs of detected changes using estimated homography and the Hungarian algorithm.
  arXiv  Detail & Related papers  (2025-01-09T20:02:10Z)
• Gaga: Group Any Gaussians via 3D-aware Memory Bank [66.54280093684427]
  Gaga reconstructs and segments open-world 3D scenes by leveraging inconsistent 2D masks predicted by zero-shot segmentation models. By eliminating the assumption of continuous view changes in training images, Gaga demonstrates robustness to variations in camera poses. Gaga performs favorably against state-of-the-art methods, emphasizing its potential for real-world applications.
  arXiv  Detail & Related papers  (2024-04-11T17:57:19Z)
• Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments [20.890476387720483]
  MoRE is a novel approach for multi-object relocalization and reconstruction in evolving environments. We view these environments as "living scenes" and consider the problem of transforming scans taken at different points in time into a 3D reconstruction of the object instances.
  arXiv  Detail & Related papers  (2023-12-14T17:09:57Z)
• C-NERF: Representing Scene Changes as Directional Consistency Difference-based NeRF [3.0023333254953406]
  We aim to detect the changes caused by object variations in a scene represented by the neural radiance fields (NeRFs) Given an arbitrary view and two sets of scene images captured at different timestamps, we can predict the scene changes in that view. Our approach surpasses state-of-the-art 2D change detection and NeRF-based methods by a significant margin.
  arXiv  Detail & Related papers  (2023-12-05T13:27:12Z)
• SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes [59.23385953161328]
  Novel view synthesis for dynamic scenes is still a challenging problem in computer vision and graphics. We propose a new representation that explicitly decomposes the motion and appearance of dynamic scenes into sparse control points and dense Gaussians. Our method can enable user-controlled motion editing while retaining high-fidelity appearances.
  arXiv  Detail & Related papers  (2023-12-04T11:57:14Z)
• Exchange means change: an unsupervised single-temporal change detection framework based on intra- and inter-image patch exchange [44.845959222180866]
  We propose an unsupervised single-temporal CD framework based on intra- and inter-image patch exchange (I3PE) The I3PE framework allows for training deep change detectors on unpaired and unlabeled single-temporal remote sensing images. I3PE outperforms representative unsupervised approaches and achieves F1 value improvements of 10.65% and 6.99% to the SOTA method.
  arXiv  Detail & Related papers  (2023-10-01T14:50:54Z)
• The Change You Want to See [91.3755431537592]
  Given two images of the same scene, being able to automatically detect the changes in them has practical applications in a variety of domains. We tackle the change detection problem with the goal of detecting "object-level" changes in an image pair despite differences in their viewpoint and illumination.
  arXiv  Detail & Related papers  (2022-09-28T18:10:09Z)
• Active Gaze Control for Foveal Scene Exploration [124.11737060344052]
  We propose a methodology to emulate how humans and robots with foveal cameras would explore a scene. The proposed method achieves an increase in detection F1-score of 2-3 percentage points for the same number of gaze shifts.
  arXiv  Detail & Related papers  (2022-08-24T14:59:28Z)
• City-scale Scene Change Detection using Point Clouds [71.73273007900717]
  We propose a method for detecting structural changes in a city using images captured from mounted cameras over two different times. A direct comparison of the two point clouds for change detection is not ideal due to inaccurate geo-location information. To circumvent this problem, we propose a deep learning-based non-rigid registration on the point clouds. Experiments show that our method is able to detect scene changes effectively, even in the presence of viewpoint and illumination differences.
  arXiv  Detail & Related papers  (2021-03-26T08:04:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.