iSDF: Real-Time Neural Signed Distance Fields for Robot Perception

• URL: http://arxiv.org/abs/2204.02296v1
• Date: Tue, 5 Apr 2022 15:48:39 GMT
• Title: iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
• Authors: Joseph Ortiz, Alexander Clegg, Jing Dong, Edgar Sucar, David Novotny, Michael Zollhoefer, Mustafa Mukadam
• Abstract summary: iSDF is a continuous learning system for real-time signed distance field reconstruction. It produces more accurate reconstructions and better approximations of collision costs and gradients.
• Score: 64.80458128766254
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We present iSDF, a continual learning system for real-time signed distance field (SDF) reconstruction. Given a stream of posed depth images from a moving camera, it trains a randomly initialised neural network to map input 3D coordinate to approximate signed distance. The model is self-supervised by minimising a loss that bounds the predicted signed distance using the distance to the closest sampled point in a batch of query points that are actively sampled. In contrast to prior work based on voxel grids, our neural method is able to provide adaptive levels of detail with plausible filling in of partially observed regions and denoising of observations, all while having a more compact representation. In evaluations against alternative methods on real and synthetic datasets of indoor environments, we find that iSDF produces more accurate reconstructions, and better approximations of collision costs and gradients useful for downstream planners in domains from navigation to manipulation. Code and video results can be found at our project page: https://joeaortiz.github.io/iSDF/ .

Related papers

• CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization [54.69408516025872]
  CAP-UDF is a novel method to learn consistency-aware UDF from raw point clouds. We train a neural network to gradually infer the relationship between queries and the approximated surface. We also introduce a polygonization algorithm to extract surfaces using the gradients of the learned UDF.
  arXiv  Detail & Related papers  (2022-10-06T08:51:08Z)
• Robust Change Detection Based on Neural Descriptor Fields [53.111397800478294]
  We develop an object-level online change detection approach that is robust to partially overlapping observations and noisy localization results. By associating objects via shape code similarity and comparing local object-neighbor spatial layout, our proposed approach demonstrates robustness to low observation overlap and localization noises.
  arXiv  Detail & Related papers  (2022-08-01T17:45:36Z)
• Semi-signed neural fitting for surface reconstruction from unoriented point clouds [53.379712818791894]
  We propose SSN-Fitting to reconstruct a better signed distance field. SSN-Fitting consists of a semi-signed supervision and a loss-based region sampling strategy. We conduct experiments to demonstrate that SSN-Fitting achieves state-of-the-art performance under different settings.
  arXiv  Detail & Related papers  (2022-06-14T09:40:17Z)
• Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction [2.741266294612776]
  We present a novel method for reconstructing 3D surfaces using a multi-beam imaging sonar. We integrate the intensities measured by the sonar from different viewpoints for fixed cell positions in a 3D grid. We train convolutional neural networks that allow us to predict the signed distance and direction to the nearest surface for each cell.
  arXiv  Detail & Related papers  (2022-06-06T18:24:14Z)
• A Deep Signed Directional Distance Function for Object Shape Representation [12.741811850885309]
  This paper develops a new shape model that allows novel distance views by optimizing a continuous signed directional distance function (SDDF) Unlike an SDF, which measures distance to the nearest surface in any direction, an SDDF measures distance in a given direction. Our model encodes by construction the property that SDDF values decrease linearly along the viewing direction.
  arXiv  Detail & Related papers  (2021-07-23T04:11:59Z)
• Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces [68.12457459590921]
  Reconstructing continuous surfaces from 3D point clouds is a fundamental operation in 3D geometry processing. We introduce textitNeural-Pull, a new approach that is simple and leads to high quality SDFs.
  arXiv  Detail & Related papers  (2020-11-26T23:18:10Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.