Demo-Pose: Depth-Monocular Modality Fusion For Object Pose Estimation

• URL: http://arxiv.org/abs/2603.27533v1
• Date: Sun, 29 Mar 2026 05:58:04 GMT
• Title: Demo-Pose: Depth-Monocular Modality Fusion For Object Pose Estimation
• Abstract summary: We address the challenge of category-level 9-DoF pose estimation from RGB-D input, without relying on CAD models during inference.<n>We propose DeMo-Pose, a hybrid architecture that fuses semantic features with depth-based graph convolutional representations.<n>Our approach achieves real-time inference and significantly improves over state-of-the-art methods across object categories.
• Score: 5.466547563815996
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object pose estimation is a fundamental task in 3D vision with applications in robotics, AR/VR, and scene understanding. We address the challenge of category-level 9-DoF pose estimation (6D pose + 3Dsize) from RGB-D input, without relying on CAD models during inference. Existing depth-only methods achieve strong results but ignore semantic cues from RGB, while many RGB-D fusion models underperform due to suboptimal cross-modal fusion that fails to align semantic RGB cues with 3D geometric representations. We propose DeMo-Pose, a hybrid architecture that fuses monocular semantic features with depth-based graph convolutional representations via a novel multimodal fusion strategy. To further improve geometric reasoning, we introduce a novel Mesh-Point Loss (MPL) that leverages mesh structure during training without adding inference overhead. Our approach achieves real-time inference and significantly improves over state-of-the-art methods across object categories, outperforming the strong GPV-Pose baseline by 3.2\% on 3D IoU and 11.1\% on pose accuracy on the REAL275 benchmark. The results highlight the effectiveness of depth-RGB fusion and geometry-aware learning, enabling robust category-level 3D pose estimation for real-world applications.

Related papers

• GauSSmart: Enhanced 3D Reconstruction through 2D Foundation Models and Geometric Filtering [50.675710727721786]
  We propose GauSSmart, a hybrid method that bridges 2D foundational models and 3D Gaussian Splatting reconstruction.<n>Our approach integrates established 2D computer vision techniques, including convex filtering and semantic feature supervision.<n>We validate our approach across three datasets, where GauSSmart consistently outperforms existing Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-10-16T03:38:26Z)
• DKPMV: Dense Keypoints Fusion from Multi-View RGB Frames for 6D Pose Estimation of Textureless Objects [18.011730388391232]
  We propose DKPMV, a pipeline that achieves dense keypoint-level fusion.<n>We enhance the keypoint network with attentional aggregation and symmetry-aware training.<n>Experiments on the ROBI dataset demonstrate that DKPMV outperforms state-of-the-art multi-view RGB approaches.
  arXiv  Detail & Related papers  (2025-10-13T02:45:55Z)
• UniPose: Unified Cross-modality Pose Prior Propagation towards RGB-D data for Weakly Supervised 3D Human Pose Estimation [14.52285662885727]
  We present UniPose, a unified cross-modality pose prior propagation method for weakly supervised 3D human pose estimation.<n>UniPose transfers 2D annotations from large-scale RGB datasets to the 3D domain via self-supervised learning.<n> Experiments on CMU Panoptic and ITOP datasets show that UniPose achieves comparable performance to fully supervised methods.
  arXiv  Detail & Related papers  (2025-09-27T15:49:30Z)
• HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB images [27.025336665386735]
  We propose a robust, keypoint detector-free approach to estimating hand-object 3D transformations from monocular motion video/images.<n>We further integrate this with a multi-view reconstruction pipeline to accurately recover hand-object 3D shape.<n>Our method, named HOSt3R, is unconstrained, does not rely on pre-scanned object templates or camera intrinsics, and reaches state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-08-22T15:30:40Z)
• Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference [65.42565481489132]
  Humans can easily deduce the relative pose of a previously unseen object, without labeling or training, given only a single query-reference image pair.<n>We propose a novel 3D generalizable relative pose estimation method by elaborating 3D/2.5D shape perception with a 2.5D shape from RGB-D reference.<n>Our differentiable takes the 2.5D rotatable mesh textured by the RGB and the semantic maps (obtained by DINOv2 from the RGB input), then renders new RGB and semantic maps under a novel rotated view.
  arXiv  Detail & Related papers  (2024-06-26T16:01:10Z)
• MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images [57.71600854525037]
  We propose a Fuse-Describe-Match strategy for 6D pose estimation from RGB-D images. MatchU is a generic approach that fuses 2D texture and 3D geometric cues for 6D pose prediction of unseen objects.
  arXiv  Detail & Related papers  (2024-03-03T14:01:03Z)
• Towards Two-view 6D Object Pose Estimation: A Comparative Study on Fusion Strategy [16.65699606802237]
  Current RGB-based 6D object pose estimation methods have achieved noticeable performance on datasets and real world applications. This paper proposes a framework for 6D object pose estimation that learns implicit 3D information from 2 RGB images.
  arXiv  Detail & Related papers  (2022-07-01T08:22:34Z)
• Learning Stereopsis from Geometric Synthesis for 6D Object Pose Estimation [11.999630902627864]
  Current monocular-based 6D object pose estimation methods generally achieve less competitive results than RGBD-based methods. This paper proposes a 3D geometric volume based pose estimation method with a short baseline two-view setting. Experiments show that our method outperforms state-of-the-art monocular-based methods, and is robust in different objects and scenes.
  arXiv  Detail & Related papers  (2021-09-25T02:55:05Z)
• Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection [70.71934539556916]
  We learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection. Specifically, a principled geometry formula with projective modeling of 2D and 3D depth predictions in the monocular 3D object detection network is devised. Our method remarkably improves the detection performance of the state-of-the-art monocular-based method without extra data by 2.80% on the moderate test setting.
  arXiv  Detail & Related papers  (2021-07-29T12:30:39Z)
• Synthetic Training for Monocular Human Mesh Recovery [100.38109761268639]
  This paper aims to estimate 3D mesh of multiple body parts with large-scale differences from a single RGB image. The main challenge is lacking training data that have complete 3D annotations of all body parts in 2D images. We propose a depth-to-scale (D2S) projection to incorporate the depth difference into the projection function to derive per-joint scale variants.
  arXiv  Detail & Related papers  (2020-10-27T03:31:35Z)
• Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild [96.09941542587865]
  We present a novel 3D pose refinement approach based on differentiable rendering for objects of arbitrary categories in the wild. In this way, we precisely align 3D models to objects in RGB images which results in significantly improved 3D pose estimates. We evaluate our approach on the challenging Pix3D dataset and achieve up to 55% relative improvement compared to state-of-the-art refinement methods in multiple metrics.
  arXiv  Detail & Related papers  (2020-07-17T12:34:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.