Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes

• URL: http://arxiv.org/abs/2208.05067v1
• Date: Tue, 9 Aug 2022 22:56:33 GMT
• Title: Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes
• Authors: Binbin Xu, Andrew J. Davison, Stefan Leutenegger
• Abstract summary: We propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.
• Score: 30.500198859451434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior with the aim that completed object geometry leads to better object reconstruction and tracking accuracy. For each incoming RGB-D frame, we perform instance segmentation to detect objects and build data associations between the detection and the existing object maps. A new object map will be created for each unmatched detection. For each matched object, we jointly optimise its pose and latent geometry representations using geometric residual and differential rendering residual towards its shape prior and completed geometry. Our approach shows better tracking and reconstruction performance compared to methods using traditional volumetric mapping or learned shape prior approaches. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.

Related papers

• Self-supervised 3D Point Cloud Completion via Multi-view Adversarial Learning [61.14132533712537]
  We propose MAL-SPC, a framework that effectively leverages both object-level and category-specific geometric similarities to complete missing structures. Our MAL-SPC does not require any 3D complete supervision and only necessitates a single partial point cloud for each object.
  arXiv  Detail & Related papers  (2024-07-13T06:53:39Z)
• ICGNet: A Unified Approach for Instance-Centric Grasping [42.92991092305974]
  We introduce an end-to-end architecture for object-centric grasping. We show the effectiveness of the proposed method by extensively evaluating it against state-of-the-art methods on synthetic datasets.
  arXiv  Detail & Related papers  (2024-01-18T12:41:41Z)
• Loop Closure Detection Based on Object-level Spatial Layout and Semantic Consistency [14.694754836704819]
  We present an object-based loop closure detection method based on the spatial layout and semanic consistency of the 3D scene graph. Experimental results demonstrate that our proposed data association approach can construct more accurate 3D semantic maps.
  arXiv  Detail & Related papers  (2023-04-11T11:20:51Z)
• Category-level Shape Estimation for Densely Cluttered Objects [94.64287790278887]
  We propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy.
  arXiv  Detail & Related papers  (2023-02-23T13:00:17Z)
• Combining Local and Global Pose Estimation for Precise Tracking of Similar Objects [2.861848675707602]
  We present a multi-object 6D detection and tracking pipeline for potentially similar and non-textured objects. A new network architecture, trained solely with synthetic images, allows simultaneous pose estimation of multiple objects. We show how the system can be used in a real AR assistance application within the field of construction.
  arXiv  Detail & Related papers  (2022-01-31T14:36:57Z)
• Object-Augmented RGB-D SLAM for Wide-Disparity Relocalisation [3.888848425698769]
  We propose a novel object-augmented RGB-D SLAM system that is capable of constructing a consistent object map and performing relocalisation based on centroids of objects in the map.
  arXiv  Detail & Related papers  (2021-08-05T11:02:25Z)
• TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction [57.1209039399599]
  We propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation.
  arXiv  Detail & Related papers  (2021-05-16T16:15:05Z)
• From Points to Multi-Object 3D Reconstruction [71.17445805257196]
  We propose a method to detect and reconstruct multiple 3D objects from a single RGB image. A keypoint detector localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes. The presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable.
  arXiv  Detail & Related papers  (2020-12-21T18:52:21Z)
• Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstruction [79.98689027127855]
  We propose a new representation of the 3D shape of common object categories that can be learned from a collection of 2D images of independent objects. Our method builds in a novel way on concepts from parametric deformation models, non-parametric 3D reconstruction, and canonical embeddings. It achieves state-of-the-art results in dense 3D reconstruction on public in-the-wild datasets of faces, cars, and birds.
  arXiv  Detail & Related papers  (2020-08-28T15:44:05Z)
• Category Level Object Pose Estimation via Neural Analysis-by-Synthesis [64.14028598360741]
  In this paper we combine a gradient-based fitting procedure with a parametric neural image synthesis module. The image synthesis network is designed to efficiently span the pose configuration space. We experimentally show that the method can recover orientation of objects with high accuracy from 2D images alone.
  arXiv  Detail & Related papers  (2020-08-18T20:30:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.