DORec: Decomposed Object Reconstruction Utilizing 2D Self-Supervised
Features
- URL: http://arxiv.org/abs/2310.11092v2
- Date: Thu, 19 Oct 2023 14:16:49 GMT
- Title: DORec: Decomposed Object Reconstruction Utilizing 2D Self-Supervised
Features
- Authors: Jun Wu, Sicheng Li, Sihui Ji, Yue Wang, Rong Xiong, and Yiyi Liao
- Abstract summary: We propose a Decomposed Object Reconstruction network based on neural implicit representations.
Our key idea is to transfer 2D self-supervised features into masks of two levels of granularity to supervise the decomposition.
Experimental results show the superiority of DORec in segmenting and reconstructing the foreground object on various datasets.
- Score: 28.446955045371737
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Decomposing a target object from a complex background while reconstructing is
challenging. Most approaches acquire the perception for object instances
through the use of manual labels, but the annotation procedure is costly. The
recent advancements in 2D self-supervised learning have brought new prospects
to object-aware representation, yet it remains unclear how to leverage such
noisy 2D features for clean decomposition. In this paper, we propose a
Decomposed Object Reconstruction (DORec) network based on neural implicit
representations. Our key idea is to transfer 2D self-supervised features into
masks of two levels of granularity to supervise the decomposition, including a
binary mask to indicate the foreground regions and a K-cluster mask to indicate
the semantically similar regions. These two masks are complementary to each
other and lead to robust decomposition. Experimental results show the
superiority of DORec in segmenting and reconstructing the foreground object on
various datasets.
Related papers
- MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders [93.87585467898252]
We design MonoMAE, a monocular 3D detector inspired by Masked Autoencoders.
MonoMAE consists of two novel designs. The first is depth-aware masking that selectively masks certain parts of non-occluded object queries.
The second is lightweight query completion that works with the depth-aware masking to learn to reconstruct and complete the masked object queries.
arXiv Detail & Related papers (2024-05-13T12:32:45Z) - UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
We propose UGMAE, a unified framework for graph masked autoencoders.
We first develop an adaptive feature mask generator to account for the unique significance of nodes.
We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
arXiv Detail & Related papers (2024-02-12T19:39:26Z) - Iterative Superquadric Recomposition of 3D Objects from Multiple Views [77.53142165205283]
We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views.
Our framework iteratively adds new superquadrics wherever the reconstruction error is high.
It provides consistently more accurate 3D reconstructions, even from images in the wild.
arXiv Detail & Related papers (2023-09-05T10:21:37Z) - AutoRecon: Automated 3D Object Discovery and Reconstruction [41.60050228813979]
We propose a novel framework named AutoRecon for the automated discovery and reconstruction of an object from multi-view images.
We demonstrate that foreground objects can be robustly located and segmented from SfM point clouds by leveraging self-supervised 2D vision transformer features.
Experiments on the DTU, BlendedMVS and CO3D-V2 datasets demonstrate the effectiveness and robustness of AutoRecon.
arXiv Detail & Related papers (2023-05-15T17:16:46Z) - Topologically Persistent Features-based Object Recognition in Cluttered
Indoor Environments [1.2691047660244335]
Recognition of occluded objects in unseen indoor environments is a challenging problem for mobile robots.
This work proposes a new slicing-based topological descriptor that captures the 3D shape of object point clouds.
It yields similarities between the descriptors of the occluded and the corresponding unoccluded objects, enabling object unity-based recognition.
arXiv Detail & Related papers (2022-05-16T07:01:16Z) - Robust Person Re-Identification through Contextual Mutual Boosting [77.1976737965566]
We propose the Contextual Mutual Boosting Network (CMBN) to localize pedestrians.
It localizes pedestrians and recalibrates features by effectively exploiting contextual information and statistical inference.
Experiments on the benchmarks demonstrate the superiority of the architecture compared the state-of-the-art.
arXiv Detail & Related papers (2020-09-16T06:33:35Z) - AutoSweep: Recovering 3D Editable Objectsfrom a Single Photograph [54.701098964773756]
We aim to recover 3D objects with semantic parts and can be directly edited.
Our work makes an attempt towards recovering two types of primitive-shaped objects, namely, generalized cuboids and generalized cylinders.
Our algorithm can recover high quality 3D models and outperforms existing methods in both instance segmentation and 3D reconstruction.
arXiv Detail & Related papers (2020-05-27T12:16:24Z) - Disassembling Object Representations without Labels [75.2215716328001]
We study a new representation-learning task, which we termed as disassembling object representations.
Disassembling enables category-specific modularity in the learned representations.
We propose an unsupervised approach to achieving disassembling, named Unsupervised Disassembling Object Representation (UDOR)
arXiv Detail & Related papers (2020-04-03T08:23:09Z) - Object-Centric Image Generation with Factored Depths, Locations, and
Appearances [30.541425619507184]
We present a generative model of images that explicitly reasons over the set of objects they show.
Our model learns a structured latent representation that separates objects from each other and from the background.
It can be trained from images alone in a purely unsupervised fashion without the need for object masks or depth information.
arXiv Detail & Related papers (2020-04-01T18:00:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.