OCH3R: Object-Centric Holistic 3D Reconstruction

• URL: http://arxiv.org/abs/2605.13018v1
• Date: Wed, 13 May 2026 05:17:51 GMT
• Title: OCH3R: Object-Centric Holistic 3D Reconstruction
• Abstract summary: We introduce OCH3R, a unified framework for Object-Centric Holistic 3D Reconstruction from a single RGB image.<n>OCH3R performs one forward pass to simultaneously predict all object instances with their 6D poses and detailed 3D reconstructions.<n>On standard indoor benchmarks, OCH3R achieves state-of-the-art performance across monocular depth estimation, open-vocabulary semantic segmentation, and RGB-only category-level 6D pose estimation.
• Score: 36.45885767128656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object-centric scene understanding is a fundamental challenge in computer vision. Existing approaches often rely on multi-stage pipelines that first apply pre-trained segmentors to extract individual objects, followed by per-object 3D reconstruction. Such methods are computationally expensive, fragile to segmentation errors, and scale poorly with scene complexity. We introduce OCH3R, a unified framework for Object-Centric Holistic 3D Reconstruction from a single RGB image. OCH3R performs one forward pass to simultaneously predict all object instances with their 6D poses and detailed 3D reconstructions. The key idea is a transformer architecture that predicts per-pixel attributes, including CLIP-based category embeddings, metric depth, normalized object coordinates (NOCS), and a fixed number of 3D Gaussians representing each object. To supervise these Gaussian reconstructions, we transform them into canonical space using the predicted 6D poses and align them with pre-rendered canonical ground truth, avoiding costly per-image Gaussian label generation. On standard indoor benchmarks, OCH3R achieves state-of-the-art performance across monocular depth estimation, open-vocabulary semantic segmentation, and RGB-only category-level 6D pose estimation, while producing high-fidelity, editable per-object reconstructions. Crucially, inference is fully feed-forward and scales independently of the number of objects, offering orders-of-magnitude speedups over conventional multi-stage pipelines in cluttered scenes.

Related papers

• PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects [36.94429692322632]
  PIXIE is a zero-shot framework that estimates the 6D pose of an object from an RGB image using only an untextured 3D model.<n>We evaluate on widely-used public benchmarks, reporting results on texture-less objects without object-specific training, and introduce a novel dataset to demonstrate real-world applicability.
  arXiv  Detail & Related papers  (2026-07-17T14:48:43Z)
• MooMIns -- Monocular 3D Reconstruction and Object Pose Estimation from Multiple Instances [3.8233569758620054]
  Simultaneous 3D reconstruction and 6D object pose estimation from a single monocular image is an inherently ill-posed problem.<n>In industrial settings, multiple instances of an object are often randomly arranged in bins, implicitly providing several views of the same object within a single image.<n>We show that this implicit multi-view geometry can be exploited to simultaneously reconstruct the object in 3D and estimate the 6D pose of each visible object instance.
  arXiv  Detail & Related papers  (2026-06-12T12:24:50Z)
• FurnSet: Exploiting Repeats for 3D Scene Reconstruction [3.948390930350905]
  Single-view 3D scene reconstruction involves inferring both object geometry and spatial layout.<n>Existing methods typically reconstruct objects independently or rely on implicit scene context.<n>We propose FurnSet, a framework that explicitly identifies and leverages repeated object instances to improve reconstruction.
  arXiv  Detail & Related papers  (2026-04-22T01:27:13Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB images [27.025336665386735]
  We propose a robust, keypoint detector-free approach to estimating hand-object 3D transformations from monocular motion video/images.<n>We further integrate this with a multi-view reconstruction pipeline to accurately recover hand-object 3D shape.<n>Our method, named HOSt3R, is unconstrained, does not rely on pre-scanned object templates or camera intrinsics, and reaches state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-08-22T15:30:40Z)
• Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations [114.57192386025373]
  Object-X is a versatile multi-modal 3D representation framework.<n>It can encoding rich object embeddings and decoding them back into geometric and visual reconstructions.<n>It supports a range of downstream tasks, including scene alignment, single-image 3D object reconstruction, and localization.
  arXiv  Detail & Related papers  (2025-06-05T09:14:42Z)
• FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction [69.63414788486578]
  FreeSplatter is a scalable feed-forward framework that generates high-quality 3D Gaussians from uncalibrated sparse-view images.<n>Our approach employs a streamlined transformer architecture where self-attention blocks facilitate information exchange.<n>We develop two specialized variants--for object-centric and scene-level reconstruction--trained on comprehensive datasets.
  arXiv  Detail & Related papers  (2024-12-12T18:52:53Z)
• Iterative Superquadric Recomposition of 3D Objects from Multiple Views [77.53142165205283]
  We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views. Our framework iteratively adds new superquadrics wherever the reconstruction error is high. It provides consistently more accurate 3D reconstructions, even from images in the wild.
  arXiv  Detail & Related papers  (2023-09-05T10:21:37Z)
• Variable Radiance Field for Real-World Category-Specific Reconstruction from Single Image [25.44715538841181]
  Reconstructing category-specific objects using Neural Radiance Field (NeRF) from a single image is a promising yet challenging task.<n>We propose Variable Radiance Field (VRF), a novel framework capable of efficiently reconstructing category-specific objects.<n>VRF achieves state-of-the-art performance in both reconstruction quality and computational efficiency.
  arXiv  Detail & Related papers  (2023-06-08T12:12:02Z)
• RelPose++: Recovering 6D Poses from Sparse-view Observations [66.6922660401558]
  We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images) We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories.
  arXiv  Detail & Related papers  (2023-05-08T17:59:58Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• Single View Metrology in the Wild [94.7005246862618]
  We present a novel approach to single view metrology that can recover the absolute scale of a scene represented by 3D heights of objects or camera height above the ground. Our method relies on data-driven priors learned by a deep network specifically designed to imbibe weakly supervised constraints from the interplay of the unknown camera with 3D entities such as object heights. We demonstrate state-of-the-art qualitative and quantitative results on several datasets as well as applications including virtual object insertion.
  arXiv  Detail & Related papers  (2020-07-18T22:31:33Z)
• CoReNet: Coherent 3D scene reconstruction from a single RGB image [43.74240268086773]
  We build on advances in deep learning to reconstruct the shape of a single object given only one RBG image as input. We propose three extensions: (1) ray-traced skip connections that propagate local 2D information to the output 3D volume in a physically correct manner; (2) a hybrid 3D volume representation that enables building translation equivariant models; and (3) a reconstruction loss tailored to capture overall object geometry. We reconstruct all objects jointly in one pass, producing a coherent reconstruction, where all objects live in a single consistent 3D coordinate frame relative to the camera and they do not intersect in 3D space.
  arXiv  Detail & Related papers  (2020-04-27T17:53:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.