Objects Can Move: 3D Change Detection by Geometric Transformation Constistency

• URL: http://arxiv.org/abs/2208.09870v1
• Date: Sun, 21 Aug 2022 11:32:47 GMT
• Title: Objects Can Move: 3D Change Detection by Geometric Transformation Constistency
• Authors: Aikaterini Adam, Torsten Sattler, Konstantinos Karantzalos and Tomas Pajdla
• Abstract summary: AR/VR applications and robots need to know when the scene has changed. We propose a 3D object discovery method that is based only on scene changes. Our method does not need to encode any assumptions about what is an object, but rather discovers objects by exploiting their coherent move.
• Score: 32.07372152138814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AR/VR applications and robots need to know when the scene has changed. An example is when objects are moved, added, or removed from the scene. We propose a 3D object discovery method that is based only on scene changes. Our method does not need to encode any assumptions about what is an object, but rather discovers objects by exploiting their coherent move. Changes are initially detected as differences in the depth maps and segmented as objects if they undergo rigid motions. A graph cut optimization propagates the changing labels to geometrically consistent regions. Experiments show that our method achieves state-of-the-art performance on the 3RScan dataset against competitive baselines. The source code of our method can be found at https://github.com/katadam/ObjectsCanMove.

Related papers

• SPOC: Spatially-Progressing Object State Change Segmentation in Video [52.65373395382122]
  We introduce the spatially-progressing object state change segmentation task. The goal is to segment at the pixel-level those regions of an object that are actionable and those that are transformed. We demonstrate useful implications for tracking activity progress to benefit robotic agents.
  arXiv  Detail & Related papers  (2025-03-15T01:48:54Z)
• Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs [44.14584011692035]
  Static semantic maps are unable to capture interactions between the environment and humans or robotic agents. We present an approach that addresses this limitation. Based solely on egocentric recordings, we are able to track the 6DoF poses of the moving object. We show how our method allows to command a mobile manipulator through teach & repeat, and how information about prior interaction allows a mobile manipulator to retrieve an object hidden in a drawer.
  arXiv  Detail & Related papers  (2024-11-28T14:05:07Z)
• PickScan: Object discovery and reconstruction from handheld interactions [99.99566882133179]
  We develop an interaction-guided and class-agnostic method to reconstruct 3D representations of scenes. Our main contribution is a novel approach to detecting user-object interactions and extracting the masks of manipulated objects. Compared to Co-Fusion, the only comparable interaction-based and class-agnostic baseline, this corresponds to a reduction in chamfer distance of 73%.
  arXiv  Detail & Related papers  (2024-11-17T23:09:08Z)
• 3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement [2.2122801766964795]
  We present 3DGS-CD, the first 3D Gaussian Splatting (3DGS)-based method for detecting physical object rearrangements in 3D scenes. Our approach estimates 3D object-level changes by comparing two sets of unaligned images taken at different times. Our method can detect changes in cluttered environments using sparse post-change images within as little as 18s, using as few as a single new image.
  arXiv  Detail & Related papers  (2024-11-06T07:08:41Z)
• A3D: Does Diffusion Dream about 3D Alignment? [73.97853402817405]
  We tackle the problem of text-driven 3D generation from a geometry alignment perspective. Given a set of text prompts, we aim to generate a collection of objects with semantically corresponding parts aligned across them. We propose to embed these objects into a common latent space and optimize the continuous transitions between these objects.
  arXiv  Detail & Related papers  (2024-06-21T09:49:34Z)
• GeoDiffuser: Geometry-Based Image Editing with Diffusion Models [7.7669649283012]
  We present GeoDiffuser, a zero-shot optimization-based method that unifies 2D and 3D image-based object editing capabilities into a single method. We show that these transformations can be directly incorporated into the attention layers in diffusion models to implicitly perform editing operations. GeoDiffuser can perform common 2D and 3D edits like object translation, 3D rotation, and removal.
  arXiv  Detail & Related papers  (2024-04-22T17:58:36Z)
• 3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection [35.61749990140511]
  A major challenge in monocular 3D object detection is the limited diversity and quantity of objects in real datasets. We propose a physically plausible indoor 3D object insertion approach to automatically copy virtual objects and paste them into real scenes.
  arXiv  Detail & Related papers  (2023-12-08T08:44:54Z)
• Has Anything Changed? 3D Change Detection by 2D Segmentation Masks [27.15724607877779]
  3D scans of interior spaces are acquired on a daily basis. This information is important for robots and AR and VR devices, in order to operate in an immersive virtual experience. We propose an unsupervised object discovery method that identifies added, moved, or removed objects without any prior knowledge of what objects exist in the scene.
  arXiv  Detail & Related papers  (2023-12-02T14:30:23Z)
• 3D Video Object Detection with Learnable Object-Centric Global Optimization [65.68977894460222]
  Correspondence-based optimization is the cornerstone for 3D scene reconstruction but is less studied in 3D video object detection. We propose BA-Det, an end-to-end optimizable object detector with object-centric temporal correspondence learning and featuremetric object bundle adjustment.
  arXiv  Detail & Related papers  (2023-03-27T17:39:39Z)
• BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects [89.2314092102403]
  We present a near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence. Our method works for arbitrary rigid objects, even when visual texture is largely absent.
  arXiv  Detail & Related papers  (2023-03-24T17:13:49Z)
• LocPoseNet: Robust Location Prior for Unseen Object Pose Estimation [69.70498875887611]
  LocPoseNet is able to robustly learn location prior for unseen objects. Our method outperforms existing works by a large margin on LINEMOD and GenMOP.
  arXiv  Detail & Related papers  (2022-11-29T15:21:34Z)
• KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control [64.46042014759671]
  KeypointDeformer is an unsupervised method for shape control through automatically discovered 3D keypoints. Our approach produces intuitive and semantically consistent control of shape deformations.
  arXiv  Detail & Related papers  (2021-04-22T17:59:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.