Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans

• URL: http://arxiv.org/abs/2304.09704v2
• Date: Thu, 28 Mar 2024 17:53:08 GMT
• Title: Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans
• Authors: Romain Loiseau, Elliot Vincent, Mathieu Aubry, Loic Landrieu,
• Abstract summary: We propose an unsupervised method for parsing large 3D scans of real-world scenes with easily-interpretable shapes. Our approach is based on a probabilistic reconstruction model that decomposes an input 3D point cloud into a small set of learned 3D shapes. We demonstrate the usefulness of our model on a novel dataset of seven large aerial LiDAR scans from diverse real-world scenarios.
• Score: 20.030706182672144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose an unsupervised method for parsing large 3D scans of real-world scenes with easily-interpretable shapes. This work aims to provide a practical tool for analyzing 3D scenes in the context of aerial surveying and mapping, without the need for user annotations. Our approach is based on a probabilistic reconstruction model that decomposes an input 3D point cloud into a small set of learned prototypical 3D shapes. The resulting reconstruction is visually interpretable and can be used to perform unsupervised instance and low-shot semantic segmentation of complex scenes. We demonstrate the usefulness of our model on a novel dataset of seven large aerial LiDAR scans from diverse real-world scenarios. Our approach outperforms state-of-the-art unsupervised methods in terms of decomposition accuracy while remaining visually interpretable. Our code and dataset are available at https://romainloiseau.fr/learnable-earth-parser/

Related papers

• DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features [65.8738034806085]
  DistillNeRF is a self-supervised learning framework for understanding 3D environments in autonomous driving scenes. Our method is a generalizable feedforward model that predicts a rich neural scene representation from sparse, single-frame multi-view camera inputs.
  arXiv  Detail & Related papers  (2024-06-17T21:15:13Z)
• 3D LiDAR Mapping in Dynamic Environments Using a 4D Implicit Neural Representation [33.92758288570465]
  Building accurate maps is a key building block to enable reliable localization, planning, and navigation of autonomous vehicles. We propose encoding the 4D scene into a novel implicit neural map representation. Our method is capable of removing the dynamic part of the input point clouds while reconstructing accurate and complete 3D maps.
  arXiv  Detail & Related papers  (2024-05-06T11:46:04Z)
• LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset [17.530432165466507]
  We present a novel Cross-Modal Shape Reconstruction (DisCo) method and an Occupancy-Guided 3D Object Detection (OccGOD) method. Our methods achieve state-of-the-art performance in both instance-level scene reconstruction and 3D object detection tasks.
  arXiv  Detail & Related papers  (2023-12-19T18:50:10Z)
• CLIP$^2$: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data [80.42480679542697]
  We propose Contrastive Language-Image-Point Cloud Pretraining (CLIP$2$) to learn the transferable 3D point cloud representation in realistic scenarios. Specifically, we exploit naturally-existed correspondences in 2D and 3D scenarios, and build well-aligned and instance-based text-image-point proxies from those complex scenarios.
  arXiv  Detail & Related papers  (2023-03-22T09:32:45Z)
• DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension [71.71234436165255]
  We contribute DensePose 3D, a method that can learn such reconstructions in a weakly supervised fashion from 2D image annotations only. Because it does not require 3D scans, DensePose 3D can be used for learning a wide range of articulated categories such as different animal species. We show significant improvements compared to state-of-the-art non-rigid structure-from-motion baselines on both synthetic and real data on categories of humans and animals.
  arXiv  Detail & Related papers  (2021-08-31T18:33:55Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection [101.20784125067559]
  We propose a new architecture, namely Hallucinated Hollow-3D R-CNN, to address the problem of 3D object detection. In our approach, we first extract the multi-view features by sequentially projecting the point clouds into the perspective view and the bird-eye view. The 3D objects are detected via a box refinement module with a novel Hierarchical Voxel RoI Pooling operation.
  arXiv  Detail & Related papers  (2021-07-30T02:00:06Z)
• Cuboids Revisited: Learning Robust 3D Shape Fitting to Single RGB Images [44.223070672713455]
  In particular, man-made environments commonly consist of volumetric primitives such as cuboids or cylinders. Previous approaches directly estimate shape parameters from a 2D or 3D input, and are only able to reproduce simple objects. We propose a robust estimator for primitive fitting, which can meaningfully abstract real-world environments using cuboids.
  arXiv  Detail & Related papers  (2021-05-05T13:36:00Z)
• DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes [54.239416488865565]
  We propose a fast single-stage 3D object detection method for LIDAR data. The core novelty of our method is a fast, single-pass architecture that both detects objects in 3D and estimates their shapes. We find that our proposed method achieves state-of-the-art results by 5% on object detection in ScanNet scenes, and it gets top results by 3.4% in the Open dataset.
  arXiv  Detail & Related papers  (2020-04-02T17:48:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.