Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change

• URL: http://arxiv.org/abs/2311.09346v1
• Date: Wed, 15 Nov 2023 20:09:29 GMT
• Title: Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change
• Authors: Tao Sun, Yan Hao, Shengyu Huang, Silvio Savarese, Konrad Schindler, Marc Pollefeys, Iro Armeni
• Abstract summary: Building 3D geometric maps of man-made spaces are fundamental computer vision and robotics. Nothing Stands Still (NSS) benchmark focuses on thetemporal registration of 3D scenes undergoing large spatial and temporal change. As part of NSS, we introduce a dataset of 3D point clouds recurrently captured in large-scale building indoor environments that are under construction or renovation.
• Score: 86.44429778015657
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Building 3D geometric maps of man-made spaces is a well-established and active field that is fundamental to computer vision and robotics. However, considering the evolving nature of built environments, it is essential to question the capabilities of current mapping efforts in handling temporal changes. In addition, spatiotemporal mapping holds significant potential for achieving sustainability and circularity goals. Existing mapping approaches focus on small changes, such as object relocation or self-driving car operation; in all cases where the main structure of the scene remains fixed. Consequently, these approaches fail to address more radical changes in the structure of the built environment, such as geometry and topology. To this end, we introduce the Nothing Stands Still (NSS) benchmark, which focuses on the spatiotemporal registration of 3D scenes undergoing large spatial and temporal change, ultimately creating one coherent spatiotemporal map. Specifically, the benchmark involves registering two or more partial 3D point clouds (fragments) from the same scene but captured from different spatiotemporal views. In addition to the standard pairwise registration, we assess the multi-way registration of multiple fragments that belong to any temporal stage. As part of NSS, we introduce a dataset of 3D point clouds recurrently captured in large-scale building indoor environments that are under construction or renovation. The NSS benchmark presents three scenarios of increasing difficulty, to quantify the generalization ability of point cloud registration methods over space (within one building and across buildings) and time. We conduct extensive evaluations of state-of-the-art methods on NSS. The results demonstrate the necessity for novel methods specifically designed to handle large spatiotemporal changes. The homepage of our benchmark is at http://nothing-stands-still.com.

Related papers

• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments [20.890476387720483]
  MoRE is a novel approach for multi-object relocalization and reconstruction in evolving environments. We view these environments as "living scenes" and consider the problem of transforming scans taken at different points in time into a 3D reconstruction of the object instances.
  arXiv  Detail & Related papers  (2023-12-14T17:09:57Z)
• 3DInAction: Understanding Human Actions in 3D Point Clouds [31.66883982183386]
  We propose a novel method for 3D point cloud action recognition. We show that our method achieves improved performance on existing datasets, including ASM videos.
  arXiv  Detail & Related papers  (2023-03-11T08:42:54Z)
• SpOT: Spatiotemporal Modeling for 3D Object Tracking [68.12017780034044]
  3D multi-object tracking aims to consistently identify all mobile time. Current 3D tracking methods rely on abstracted information and limited history. We develop a holistic representation of scenes that leverage both spatial and temporal information.
  arXiv  Detail & Related papers  (2022-07-12T21:45:49Z)
• Learning Spatial and Temporal Variations for 4D Point Cloud Segmentation [0.39373541926236766]
  We argue that the temporal information across the frames provides crucial knowledge for 3D scene perceptions. We design a temporal variation-aware module and a temporal voxel-point refiner to capture the temporal variation in the 4D point cloud.
  arXiv  Detail & Related papers  (2022-07-11T07:36:26Z)
• PointAttN: You Only Need Attention for Point Cloud Completion [89.88766317412052]
  Point cloud completion refers to completing 3D shapes from partial 3D point clouds. We propose a novel neural network for processing point cloud in a per-point manner to eliminate kNNs. The proposed framework, namely PointAttN, is simple, neat and effective, which can precisely capture the structural information of 3D shapes.
  arXiv  Detail & Related papers  (2022-03-16T09:20:01Z)
• Spatio-Temporal Representation Factorization for Video-based Person Re-Identification [55.01276167336187]
  We propose Spatio-Temporal Representation Factorization module (STRF) for re-ID. STRF is a flexible new computational unit that can be used in conjunction with most existing 3D convolutional neural network architectures for re-ID. We empirically show that STRF improves performance of various existing baseline architectures while demonstrating new state-of-the-art results.
  arXiv  Detail & Related papers  (2021-07-25T19:29:37Z)
• CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations [72.4716073597902]
  We propose a method to learn object Canonical Point Cloud Representations of dynamically or moving objects. We demonstrate the effectiveness of our method on several applications including shape reconstruction, camera pose estimation, continuoustemporal sequence reconstruction, and correspondence estimation.
  arXiv  Detail & Related papers  (2020-08-06T17:58:48Z)
• Unsupervised Learning of Global Registration of Temporal Sequence of Point Clouds [16.019588704177288]
  Global registration of point clouds aims to find an optimal alignment of a sequence of 2D or 3D point sets. We present a novel method that takes advantage of current deep learning techniques for unsupervised learning of global registration from a temporal sequence of point clouds.
  arXiv  Detail & Related papers  (2020-06-17T06:00:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.