Learning Reconstructability for Drone Aerial Path Planning

• URL: http://arxiv.org/abs/2209.10174v1
• Date: Wed, 21 Sep 2022 08:10:26 GMT
• Title: Learning Reconstructability for Drone Aerial Path Planning
• Authors: Yilin Liu, Liqiang Lin, Yue Hu, Ke Xie, Chi-Wing Fu, Hao Zhang, Hui Huang
• Abstract summary: We introduce the first learning-based reconstructability predictor to improve view and path planning for large-scale 3D urban scene acquisition using unmanned drones. In contrast to previous approaches, our method learns a model that explicitly predicts how well a 3D urban scene will be reconstructed from a set of viewpoints.
• Score: 51.736344549907265
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce the first learning-based reconstructability predictor to improve view and path planning for large-scale 3D urban scene acquisition using unmanned drones. In contrast to previous heuristic approaches, our method learns a model that explicitly predicts how well a 3D urban scene will be reconstructed from a set of viewpoints. To make such a model trainable and simultaneously applicable to drone path planning, we simulate the proxy-based 3D scene reconstruction during training to set up the prediction. Specifically, the neural network we design is trained to predict the scene reconstructability as a function of the proxy geometry, a set of viewpoints, and optionally a series of scene images acquired in flight. To reconstruct a new urban scene, we first build the 3D scene proxy, then rely on the predicted reconstruction quality and uncertainty measures by our network, based off of the proxy geometry, to guide the drone path planning. We demonstrate that our data-driven reconstructability predictions are more closely correlated to the true reconstruction quality than prior heuristic measures. Further, our learned predictor can be easily integrated into existing path planners to yield improvements. Finally, we devise a new iterative view planning framework, based on the learned reconstructability, and show superior performance of the new planner when reconstructing both synthetic and real scenes.

Related papers

• PVP-Recon: Progressive View Planning via Warping Consistency for Sparse-View Surface Reconstruction [49.7580491592023]
  We propose PVP-Recon, a novel and effective sparse-view surface reconstruction method. PVP-Recon starts initial surface reconstruction with as few as 3 views and progressively adds new views. This progressive view planning progress is interleaved with a neural SDF-based reconstruction module.
  arXiv  Detail & Related papers  (2024-09-09T10:06:34Z)
• MaRINeR: Enhancing Novel Views by Matching Rendered Images with Nearby References [49.71130133080821]
  MaRINeR is a refinement method that leverages information of a nearby mapping image to improve the rendering of a target viewpoint. We show improved renderings in quantitative metrics and qualitative examples from both explicit and implicit scene representations.
  arXiv  Detail & Related papers  (2024-07-18T17:50:03Z)
• UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos [12.328095228008893]
  We present UniPlane, a novel method that unifies plane detection and reconstruction from posed monocular videos. We build a Transformers-based deep neural network that jointly constructs a 3D feature volume for the environment. Experiments on real-world datasets demonstrate that UniPlane outperforms state-of-the-art methods in both plane detection and reconstruction tasks.
  arXiv  Detail & Related papers  (2024-07-04T03:02:27Z)
• Exploiting Priors from 3D Diffusion Models for RGB-Based One-Shot View Planning [24.44253219419552]
  We propose a novel one-shot view planning approach that utilizes the powerful 3D generation capabilities of diffusion models as priors. Our experiments in simulation and real-world setups indicate that our approach balances well between object reconstruction quality and movement cost.
  arXiv  Detail & Related papers  (2024-03-25T14:21:49Z)
• OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments [77.0399450848749]
  We propose an OccNeRF method for training occupancy networks without 3D supervision. We parameterize the reconstructed occupancy fields and reorganize the sampling strategy to align with the cameras' infinite perceptive range. For semantic occupancy prediction, we design several strategies to polish the prompts and filter the outputs of a pretrained open-vocabulary 2D segmentation model.
  arXiv  Detail & Related papers  (2023-12-14T18:58:52Z)
• Bag of Views: An Appearance-based Approach to Next-Best-View Planning for 3D Reconstruction [3.637651065605852]
  Bag-of-Views (BoV) is a fully appearance-based model used to assign utility to captured views. View Planning Toolbox (VPT) is a lightweight package for training and testing machine learning-based view planning frameworks.
  arXiv  Detail & Related papers  (2023-07-11T22:56:55Z)
• PlaneFormers: From Sparse View Planes to 3D Reconstruction [14.45228936875838]
  We present an approach for the planar surface reconstruction of a scene from images with limited overlap. We introduce a simpler approach, the PlaneFormer, that uses a transformer applied to 3D-aware plane tokens to perform 3D reasoning.
  arXiv  Detail & Related papers  (2022-08-08T17:58:13Z)
• NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction [64.36535692191343]
  Implicit neural representations have shown compelling results in offline 3D reconstruction and also recently demonstrated the potential for online SLAM systems. This paper addresses two key challenges: 1) seeking a criterion to measure the quality of the candidate viewpoints for the view planning based on the new representations, and 2) learning the criterion from data that can generalize to different scenes instead of hand-crafting one. Our method demonstrates significant improvements on various metrics for the rendered image quality and the geometry quality of the reconstructed 3D models when compared with variants using TSDF or reconstruction without view planning.
  arXiv  Detail & Related papers  (2022-07-22T10:05:36Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.