TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

• URL: http://arxiv.org/abs/2105.07468v1
• Date: Sun, 16 May 2021 16:15:05 GMT
• Title: TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction
• Authors: Margarita Grinvald, Federico Tombari, Roland Siegwart, Juan Nieto
• Abstract summary: We propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation.
• Score: 57.1209039399599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to simultaneously track and reconstruct multiple objects moving in the scene is of the utmost importance for robotic tasks such as autonomous navigation and interaction. Virtually all of the previous attempts to map multiple dynamic objects have evolved to store individual objects in separate reconstruction volumes and track the relative pose between them. While simple and intuitive, such formulation does not scale well with respect to the number of objects in the scene and introduces the need for an explicit occlusion handling strategy. In contrast, we propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. To this end, we introduce a novel multi-object TSDF formulation that can encode multiple object surfaces at any given location in the map. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation.

Related papers

• Sparse multi-view hand-object reconstruction for unseen environments [31.604141859402187]
  We train our model on a synthetic hand-object dataset and evaluate directly on a real world recorded hand-object dataset with unseen objects. We show that while reconstruction of unseen hands and objects from RGB is challenging, additional views can help improve the reconstruction quality.
  arXiv  Detail & Related papers  (2024-05-02T15:01:25Z)
• Iterative Superquadric Recomposition of 3D Objects from Multiple Views [77.53142165205283]
  We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views. Our framework iteratively adds new superquadrics wherever the reconstruction error is high. It provides consistently more accurate 3D reconstructions, even from images in the wild.
  arXiv  Detail & Related papers  (2023-09-05T10:21:37Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)
• AssetField: Assets Mining and Reconfiguration in Ground Feature Plane Representation [111.59786941545774]
  AssetField is a novel neural scene representation that learns a set of object-aware ground feature planes to represent the scene. We show that AssetField achieves competitive performance for novel-view synthesis and generates realistic renderings for new scene configurations.
  arXiv  Detail & Related papers  (2023-03-24T12:18:10Z)
• Scene-level Tracking and Reconstruction without Object Priors [14.068026331380844]
  We present the first real-time system capable of tracking and reconstructing, individually, every visible object in a given scene. Our proposed system can provide the live geometry and deformation of all visible objects in a novel scene in real-time.
  arXiv  Detail & Related papers  (2022-10-07T20:56:14Z)
• Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes [30.500198859451434]
  We propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.
  arXiv  Detail & Related papers  (2022-08-09T22:56:33Z)
• Object-Compositional Neural Implicit Surfaces [45.274466719163925]
  The neural implicit representation has shown its effectiveness in novel view synthesis and high-quality 3D reconstruction from multi-view images. This paper proposes a novel framework, ObjectSDF, to build an object-compositional neural implicit representation with high fidelity in 3D reconstruction and object representation.
  arXiv  Detail & Related papers  (2022-07-20T06:38:04Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)
• Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments [81.38641691636847]
  We rethink the problem of scene reconstruction from an embodied agent's perspective. We reconstruct an interactive scene using RGB-D data stream. This reconstructed scene replaces the object meshes in the dense panoptic map with part-based articulated CAD models.
  arXiv  Detail & Related papers  (2021-03-30T05:56:58Z)
• MOLTR: Multiple Object Localisation, Tracking, and Reconstruction from Monocular RGB Videos [30.541606989348377]
  MOLTR is a solution to object-centric mapping using only monocular image sequences and camera poses. It is able to localise, track, and reconstruct multiple objects in an online fashion when an RGB camera captures a video of the surrounding. We evaluate localisation, tracking, and reconstruction on benchmarking datasets for indoor and outdoor scenes.
  arXiv  Detail & Related papers  (2020-12-09T23:15:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.