Scene-level Tracking and Reconstruction without Object Priors
- URL: http://arxiv.org/abs/2210.03815v1
- Date: Fri, 7 Oct 2022 20:56:14 GMT
- Title: Scene-level Tracking and Reconstruction without Object Priors
- Authors: Haonan Chang and Abdeslam Boularias
- Abstract summary: We present the first real-time system capable of tracking and reconstructing, individually, every visible object in a given scene.
Our proposed system can provide the live geometry and deformation of all visible objects in a novel scene in real-time.
- Score: 14.068026331380844
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present the first real-time system capable of tracking and reconstructing,
individually, every visible object in a given scene, without any form of prior
on the rigidness of the objects, texture existence, or object category. In
contrast with previous methods such as Co-Fusion and MaskFusion that first
segment the scene into individual objects and then process each object
independently, the proposed method dynamically segments the non-rigid scene as
part of the tracking and reconstruction process. When new measurements indicate
topology change, reconstructed models are updated in real-time to reflect that
change. Our proposed system can provide the live geometry and deformation of
all visible objects in a novel scene in real-time, which makes it possible to
be integrated seamlessly into numerous existing robotics applications that rely
on object models for grasping and manipulation. The capabilities of the
proposed system are demonstrated in challenging scenes that contain multiple
rigid and non-rigid objects.
Related papers
- ICGNet: A Unified Approach for Instance-Centric Grasping [42.92991092305974]
We introduce an end-to-end architecture for object-centric grasping.
We show the effectiveness of the proposed method by extensively evaluating it against state-of-the-art methods on synthetic datasets.
arXiv Detail & Related papers (2024-01-18T12:41:41Z) - AssetField: Assets Mining and Reconfiguration in Ground Feature Plane
Representation [111.59786941545774]
AssetField is a novel neural scene representation that learns a set of object-aware ground feature planes to represent the scene.
We show that AssetField achieves competitive performance for novel-view synthesis and generates realistic renderings for new scene configurations.
arXiv Detail & Related papers (2023-03-24T12:18:10Z) - Learning to Complete Object Shapes for Object-level Mapping in Dynamic
Scenes [30.500198859451434]
We propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes.
It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior.
We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.
arXiv Detail & Related papers (2022-08-09T22:56:33Z) - Discovering Objects that Can Move [55.743225595012966]
We study the problem of object discovery -- separating objects from the background without manual labels.
Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions.
We choose to focus on dynamic objects -- entities that can move independently in the world.
arXiv Detail & Related papers (2022-03-18T21:13:56Z) - Reconstructing Small 3D Objects in front of a Textured Background [0.0]
We present a technique for a complete 3D reconstruction of small objects moving in front of a textured background.
It is a particular variation of multibody structure from motion, which specializes to two objects only.
In experiments with real artifacts, we show that our approach has practical advantages when reconstructing 3D objects from all sides.
arXiv Detail & Related papers (2021-05-24T15:36:33Z) - TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and
Reconstruction [57.1209039399599]
We propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein.
In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity.
We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation.
arXiv Detail & Related papers (2021-05-16T16:15:05Z) - Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition [57.088328223220934]
Existing scene understanding systems mainly focus on recognizing the visible parts of a scene, ignoring the intact appearance of physical objects in the real-world.
In this work, we propose a higher-level scene understanding system to tackle both visible and invisible parts of objects and backgrounds in a given scene.
arXiv Detail & Related papers (2021-04-12T11:37:23Z) - Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model
Alignments [81.38641691636847]
We rethink the problem of scene reconstruction from an embodied agent's perspective.
We reconstruct an interactive scene using RGB-D data stream.
This reconstructed scene replaces the object meshes in the dense panoptic map with part-based articulated CAD models.
arXiv Detail & Related papers (2021-03-30T05:56:58Z) - Object-Centric Image Generation from Layouts [93.10217725729468]
We develop a layout-to-image-generation method to generate complex scenes with multiple objects.
Our method learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity.
We introduce SceneFID, an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.
arXiv Detail & Related papers (2020-03-16T21:40:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.