Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Images

• URL: http://arxiv.org/abs/2407.06984v2
• Date: Wed, 17 Jul 2024 11:13:08 GMT
• Title: Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Images
• Authors: Chuanrui Zhang, Yonggen Ling, Minglei Lu, Minghan Qin, Haoqian Wang,
• Abstract summary: Existing monocular and RGB-D methods suffer from scale ambiguity due to missing or depth measurements. We present CODERS, a one-stage approach for Category-level Object Detection, pose Estimation and Reconstruction from Stereo images. Our dataset, code, and demos will be available on our project page.
• Score: 15.921719523588996
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the 3D object understanding task for manipulating everyday objects with different material properties (diffuse, specular, transparent and mixed). Existing monocular and RGB-D methods suffer from scale ambiguity due to missing or imprecise depth measurements. We present CODERS, a one-stage approach for Category-level Object Detection, pose Estimation and Reconstruction from Stereo images. The base of our pipeline is an implicit stereo matching module that combines stereo image features with 3D position information. Concatenating this presented module and the following transform-decoder architecture leads to end-to-end learning of multiple tasks required by robot manipulation. Our approach significantly outperforms all competing methods in the public TOD dataset. Furthermore, trained on simulated data, CODERS generalize well to unseen category-level object instances in real-world robot manipulation experiments. Our dataset, code, and demos will be available on our project page.

Related papers

• Object-Oriented Material Classification and 3D Clustering for Improved Semantic Perception and Mapping in Mobile Robots [6.395242048226456]
  We propose a complement-aware deep learning approach for RGB-D-based material classification built on top of an object-oriented pipeline. We show a significant improvement in material classification and 3D clustering accuracy compared to state-of-the-art approaches for 3D semantic scene mapping.
  arXiv  Detail & Related papers  (2024-07-08T16:25:01Z)
• LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset [17.530432165466507]
  We present a novel Cross-Modal Shape Reconstruction (DisCo) method and an Occupancy-Guided 3D Object Detection (OccGOD) method. Our methods achieve state-of-the-art performance in both instance-level scene reconstruction and 3D object detection tasks.
  arXiv  Detail & Related papers  (2023-12-19T18:50:10Z)
• AutoDecoding Latent 3D Diffusion Models [95.7279510847827]
  We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core. The 3D autodecoder framework embeds properties learned from the target dataset in the latent space. We then identify the appropriate intermediate volumetric latent space, and introduce robust normalization and de-normalization operations.
  arXiv  Detail & Related papers  (2023-07-07T17:59:14Z)
• CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection [57.44434974289945]
  We propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework. Our framework takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene. In addition to 3D object detection, we investigate the effectiveness of our framework for the problem of 3D object counting.
  arXiv  Detail & Related papers  (2022-09-13T05:26:09Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• DONet: Learning Category-Level 6D Object Pose and Size Estimation from Depth Observation [53.55300278592281]
  We propose a method of Category-level 6D Object Pose and Size Estimation (COPSE) from a single depth image. Our framework makes inferences based on the rich geometric information of the object in the depth channel alone. Our framework competes with state-of-the-art approaches that require labeled real-world images.
  arXiv  Detail & Related papers  (2021-06-27T10:41:50Z)
• Stereo Object Matching Network [78.35697025102334]
  This paper presents a stereo object matching method that exploits both 2D contextual information from images and 3D object-level information. We present two novel strategies to handle 3D objectness in the cost volume space: selective sampling (RoISelect) and 2D-3D fusion.
  arXiv  Detail & Related papers  (2021-03-23T12:54:43Z)
• Info3D: Representation Learning on 3D Objects using Mutual Information Maximization and Contrastive Learning [8.448611728105513]
  We propose to extend the InfoMax and contrastive learning principles on 3D shapes. We show that we can maximize the mutual information between 3D objects and their "chunks" to improve the representations in aligned datasets.
  arXiv  Detail & Related papers  (2020-06-04T00:30:26Z)
• DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes [54.239416488865565]
  We propose a fast single-stage 3D object detection method for LIDAR data. The core novelty of our method is a fast, single-pass architecture that both detects objects in 3D and estimates their shapes. We find that our proposed method achieves state-of-the-art results by 5% on object detection in ScanNet scenes, and it gets top results by 3.4% in the Open dataset.
  arXiv  Detail & Related papers  (2020-04-02T17:48:50Z)
• Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework using Visual and Depth Cues [12.984393386954219]
  This paper addresses the problem of building augmented metric representations of scenes with semantic information from RGB-D images. We propose a complete framework to create an enhanced map representation of the environment with object-level information.
  arXiv  Detail & Related papers  (2020-03-13T15:05:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.